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ABSTRACT 


This  thesis  examines  the  termination  of  the  Navy's  A- 12  Program.  Specifically,  the 
research  sought  to  answer  the  question:  Were  the  A- 12  Program's  cost  overruns 
exceptional  when  compared  to  other  major  acquisitions?  Prior  research  indicates  that 
most  major  programs  experience  some  degree  of  cost  variance.  To  determine  if  the 
A-12's  overruns  were  exceptional,  the  A- 12  Program  and  58  other  contracts  for 
developmental  work  are  compared.  The  conclusion  of  the  research  is  the  A-12's  overruns 
were  exceptional.  The  cost  overruns  in  the  A- 12  Program,  at  termination,  exceeded  97 
percent  of  other  programs  examined.  To  complete  the  Program  may  have  cost  between  $9 
and  $1 1  billion.  The  required  budget  adjustment  to  complete  the  A- 12  Program  was 
greater  than  91  percent  of  other  programs.  The  research  found  no  difference  between 
cost  variances  of  fixed-price  contracts  and  cost-type  contracts.  The  assertion  that  the  use 
of  a  fixed-price  contract  contributed  to  the  failure  of  the  program  was  not  proven.  There 
was  also  no  statistical  difference  between  the  cost  overruns  of  aircraft  programs  and  other 
types  of  programs.  The  Government's  decision  to  terminate  the  A-12  Program  for  cost 
overruns  is  justified,  based  on  the  sample  of  programs  examined. 
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I.  INTRODUCTION 


A.  OVERVIEW 

On  7  January  1991,  Secretary  of  Defense  Richard  Cheney  announced  the 
termination  for  default  of  the  Full-Scale  Development  contract  for  the  Navy's  A- 12 
Avenger  program.  The  Secretary  of  Defense  cited  cost  overruns  and  schedule  slips  in  his 
decision  to  cancel  the  A- 12.  The  program  was  estimated  to  be  $1  billion  over  cost  and 
more  than  one  year  behind  schedule.  The  Navy's  internal  investigation  of  the  program. 

The  Beach  Report,  cited  numerous  problems  with  the  program  [Ref.  2]. 

In  the  same  year  Congressional  Oversight  Committees  reviewed  the  Air  Force's 
C-17  acquisition  program.  This  program,  like  the  A- 12,  was  a  fixed-price  contract  for 
development.  In  November  of  1991  it  was  estimated  to  be  at  least  $1  billion  over  cost  and 
one  year  behind  schedule.  While  the  A- 12  program  was  terminated,  the  C-17  continued 
into  production. 

The  Wall  Street  Journal  called  the  litigation  of  the  A- 12  program  the  most  costly 
case  in  history  with  legal  costs  alone  running  $30  million  per  year  [Ref.  28,  27  July  1994], 
At  stake  is  over  $3  billion  in  taxpayer  money,  including  $1.35  billion  in  overpayment  of 
progress  payments  and  a  contractor  claim  of  $1.9  billion.  On  9  December  1994,  U  S. 
Claims  Court  Judge  Robert  Hodges  vacated  the  Termination  for  Default  and  urged  the 
Navy  and  the  contractors  to  reach  a  settlement  [Ref.  28,  12  December  1994],  The  Navy 
has  decided  to  pursue  the  case  in  court  and  the  case  is  expected  to  be  heard  in  November 
of  1995. 

B.  PURPOSE 

1.  Thesis  Objectives 

The  intent  of  this  research  is  to  answer  a  straightforward  question:  Were  the 
Navy's  A-12  cost  overruns  exceptional  when  compared  to  other  major  acquisitions?  The 
answer  to  this  question  is  intended  to  be  simple:  yes  or  no.  It  is  apparent  from  the  history 
of  major  aircraft  acquisitions  that  cost  and  schedule  overruns  alone  do  not  account  for  the 
termination  of  programs.  This  research  examines  the  cost  variances  within  the  A-12 
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program,  in  comparison  with  other  major  acquisitions.  Cost  overruns  can  be  attributable 
to  many  factors  such  as  poor  cost  estimation,  poor  management  or  technical  difficulty. 
The  potential  for  costs  overrunning  the  ceiling  price  was  recognized  within  a  year  of  the 
start  of  Full-Scale  Development  (FSD).  According  to  an  analyst  at  the  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition  and  Technology  (OUSD  A&T),  earlier 
accounting  for  these  costs  may  have  kept  the  program  alive.  [Ref.  4] 

2.  Research  Questions 

Primary  Research  Question:  Were  the  cost  overruns  on  the  A- 12  contract 
exceptional  when  compared  to  other  major  aircraft  acquisitions  and  other  major  defense 
acquisitions? 

Subsidiary  Research  Questions: 

(1)  What  would  the  A-12  FSD  program  have  cost  if  it  continued  to  completion? 

(2)  Was  the  Secretary  of  Defense  justified  in  canceling  the  A-12  program? 

(3)  Are  there  differences  in  cost  overruns  between  fixed-price  contracts  and 
cost-type  contracts? 

(4)  Do  differences  exist  in  cost  overruns  of  aircraft  acquisitions  and  other  major 
acquisitions? 

(5)  What  is  the  average  cost  overrun  of  a  developmental  contract? 

3.  Expected  Benefit 

This  research  is  intended  to  determine  if  cost  overruns  in  the  A-12  program 
justified  the  termination  for  default  when  compared  to  other  major  aircraft  acquisition 
programs.  The  primary  benefit  of  this  study  is  to  provide  insight  into  the  effects  of  cost 
variances.  This  research  provides  the  program  manager  with  comparison  data  that  can 
provide  reference  points  for  progress. 

4.  Boundaries 

Only  contracts  for  developmental  programs  are  examined.  No  comparison  is 
attempted  to  programs  in  earlier  or  later  phases,  and  the  reader  is  cautioned  that 
techniques  used  in  this  study  may  not  be  applicable  to  other  phases  of  the  acquisition 
process.  While  there  are  studies  that  indicate  many  programs  experience  cost  and 
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schedule  variances,  this  study  specifically  examines  the  role  of  these  variances  in  the 
decision  to  terminate  the  Navy's  A- 12  Avenger  program. 

The  intent  of  this  research  is  to  examine  cost  overruns  in  the  A- 12  program  and 
provide  a  statistically  valid  finding  on  whether  the  A- 12  program  experienced 
extraordinary  cost  overruns.  The  methodology  uses  non-linear  cost  estimation  models  and 
non-parametric  statistics.  Recognizing  that  some  readers  of  this  study  may  not  be  familiar 
with  these  techniques  many  of  the  details  of  the  computations  are  not  included  in  the  text. 
Except  where  required  for  background  or  clarity,  the  detailed  methodology  may  be  found 
in  Appendix  B. 

5.  Limitations  and  Constraints 

The  contracts  included  in  this  research  had  to  meet  several  criteria.  First,  the 
contract  had  to  be  for  the  Engineering  and  Manufacturing  Development  (EMD)  phase, 
previously  designated  the  Full-Scale  Development  phase.  (For  continuity,  the  current 
designation  of  Phase  II  of  the  acquisition  process  will  be  used.)  Second,  the  program  had 
to  meet  the  requirements  to  be  an  Acquisition  Category  ID  (ACAT  ID)  program.  That  is, 
at  least  $200  million  in  Research,  Development,  Test  and  Evaluation  (RDT&E)  funds,  in 
1980  constant  dollars,  had  to  have  been  committed  to  the  program  [Ref.  7],  Third,  the 
program  must  have  achieved  a  significant  degree  of  completion.  The  significant  degree  of 
completion  was  selected  to  be  40  percent. 

Only  contracts  from  the  Defense  Acquisition  Executive  Summary  (DAES) 
database  were  examined.  Contracts  for  smaller  programs  were  not  examined. 

The  determination  of  estimates  at  completion  used  a  non-linear  cost  estimation 
model  commonly  called  the  Rayleigh  Distribution.  This  model  has  the  ability  to  provide 
estimates  at  completion  that  are  independent  of  current  budgets.  There  are  many  other 
models  that  provide  estimates  at  completion  that  were  not  examined  due  to  time 
constraints.  No  attempt  is  made  to  provide  a  general  model  for  predicting  costs  at 
completion. 

The  use  of  contractual  information  from  the  DAES  database  was  contingent  upon 
an  agreement  not  to  identify  the  contract  or  the  contractor.  Therefore,  the  contracts 
contained  in  Appendix  A  are  listed  by  number  for  identification.  Permission  from  OUSD 
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(A&T)  is  required  prior  to  release  of  the  database.  The  database  used  in  this  research  was 
transferred  to  the  Systems  Management  Department  of  the  Naval  Postgraduate  School. 

6.  Assumptions 

The  approach  used  for  this  study  was  an  examination  of  cost  data  from  multiple 
contracts  to  determine  if  there  were  significant  differences  that  would  account  for  the 
termination  of  the  A- 12  based  on  cost  variances.  Cost  information  was  obtained  from  the 
DAES  database.  Despite  the  judgmental  selection  of  the  contracts,  an  assumption  is  made 
that  the  contracts  drawn  from  the  database  are  representative  of  the  population. 

The  use  of  the  Rayleigh  Distribution  for  estimating  the  cost  at  completion  of  the 
A- 12  program  is  based  on  prior  research  that  indicates  this  model  accurately  represents 
the  expenditures  of  a  EMD  program.  It  is  assumed,  based  on  past  research,  that  the  A- 12 
program  can  be  modeled  using  the  Rayleigh  Distribution. 

The  effects  of  inflation  were  considered  during  the  research.  Inflation  would  have 
no  impact  on  the  cost  variance  between  Budgeted  Cost  of  Work  Performed  (BCWP)  and 
the  Actual  Cost  of  Work  Performed  (ACWP).  If  these  two  figures  were  inflated  to  a  base 
year  amount  the  cost  variance  would  remain  the  same.  The  second  area  inflation  was 
considered  is  in  determining  adjustments  to  the  Contract  Budget  Base.  An  assumption  is 
made  that  contractors  base  the  Budgeted  Cost  of  Work  Scheduled  (BCWS)  and  the 
Estimate  At  Completion  (EAC)  on  predictions  of  future  price  escalation.  This  assumption 
was  confirmed  by  a  prior  program  manager  for  Lockheed  and  by  a  cost  analyst  with  the 
Cost  Analysis  Improvement  Group  (CAIG).  No  adjustments  are  made  for  inflation. 

C.  METHODOLOGY 

1.  A-12  Cost  Estimation 

The  estimates  at  completion  for  the  A-12  were  developed  using  a  variant  of  the 
Rayleigh  Distribution  model.  This  model  is  defined  in  detail  in  Chapter  II.  The  decision 
to  use  the  Rayleigh  model  to  estimate  the  completion  cost  is  based  on  prior  research 
conducted  by  Abemethy  [Ref.  1],  Elrod  [Ref.  11],  Lee  [Ref.  18],  Gallagher  [Ref.  14]  and 
Watkins  [Ref.  29], 
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2.  The  Contract  Budget  Base 

The  use  of  cost  variances  alone  does  not  provide  a  complete  picture.  A  program 
that  experiences  significant  cost  variances  will  need  an  adjustment  to  its  Contract  Budget 
Base  (CBB)  if  it  is  to  continue.  Ideally,  there  would  be  no  cost  variances  within  a 
program  and  no  requirement  to  adjust  the  CBB.  Both  cost  variances  and  CBB 
adjustments  are  examined  in  this  research. 

3.  Data  Sources 

The  data  used  for  this  study  were  drawn  from  58  separate  contracts  for 
developmental  work.  Contract  information  was  obtained  from  the  OUSD  (A&T). 
Supporting  data  were  drawn  from  a  variety  of  sources  as  listed  in  the  references. 

4.  Hypotheses 

a.  Hypothesis  1:  Cost  overruns  in  the  A-12  Program  were  no  greater  than 
the  average  cost  overruns  in  the  sample.  This  hypothesis  was  tested  to  determine  if  the 
A-12  cost  variances  and  CBB  adjustments  were  statistically  different  than  other  major 
programs. 

b.  Hypothesis  2:  pfp  =  pc .  There  are  no  differences  in  the  cost  variances  of 
fixed-price  contracts  and  cost-type  contracts. 

c.  Hypothesis  3:  pacft  =  p^^  .  There  are  no  differences  between  cost 
variances  of  contracts  for  aircraft  and  other  contracts  in  the  sample. 

d.  Hypothesis  4:  pcbbfp  =  pcbbc .  There  are  no  differences  in  the 
adjustments  to  the  Contract  Budget  Base  for  fixed-price  contracts  and  the  adjustments  to 

the  Contract  Budget  Base  for  cost-type  contracts. 

e.  Hypothesis  5:  pcbbacft  =  pcbbsample .  There  are  no  differences  in  the 
adjustments  to  the  Contract  Budget  Base  of  aircraft  programs  and  other  programs  in  the 

sample. 

Where  psample  =  the  mean  of  the  sample  of  ACAT  ID  contracts.  pacft  =  the  mean  of 
aircraft  contracts,  p^  =  the  mean  of  fixed-price  contracts,  and  pc  =  the  mean  of  cost 

contracts.  Hypotheses  four  and  five  include  the  subscript  CBB.  Hypotheses  four  and  five 
tested  the  contract  budget  base  adjustments  from  the  indicated  samples. 

5.  Procedure  to  Test  the  Hypotheses 

a.  Hypothesis  1  was  tested  using  the  data  from  the  A-12  reports  and  the 
data  from  the  sample  contracts.  The  data  were  measured  at  completion  points  from  0  to 
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100  percent  complete  measured  by  dollars  expended.  The  A- 12  data  will  be  compared 
based  on  percentile  rankings. 

b.  Hypotheses  2  thru  5  were  tested  using  non-parametric  statistics.  The 
Mann- Whitney  test  was  chosen  for  its  ease  of  use.  The  data  do  not  support  an  assumption 
of  normality  that  would  allow  a  t-test.  Alpha  was  set  at  .05. 

6.  Justification  of  Methodology 

Prior  research  confirmed  that  the  Rayleigh  Distribution  patterns  the  expenditures  in 
developmental  programs.  Elrod  [Ref.  1 1]  tested  the  ability  of  the  Rayleigh  Distribution  to 
provide  estimates  at  completion  for  aircraft  programs  and  found  that  the  Rayleigh 
Distribution  can  be  used  for  this  purpose.  Lee  [Ref.  18]  developed  a  technique  using  the 
Rayleigh  model  to  determine  budgets  for  developmental  programs  Gallagher  [Ref.  14] 
developed  a  methodology  that  provided  probabilities  of  ending  cost  estimates  for 
developmental  programs  based  on  the  Rayleigh  Distribution 

The  cost  data  from  the  A- 12  program  were  used  to  determine  a  percentile  value 
based  on  the  distribution  that  best  patterns  the  data.  BestFit,  a  commercial  statistical 
software  package  was  used  to  determine  the  distributions  based  on  Chi-squared  values. 

The  software  evaluates  the  data  versus  18  distributions  to  provide  a  range  of  possible 
distributions  that  model  the  data.  Appendix  B  contains  complete  information  on  the 
modeling  of  the  data. 

Testing  non-normal  distributed  data  requires  tests  not  dependent  on  the  parameters 
of  the  distribution.  A  non-parametric  test  is  required  for  testing  hypotheses  on  cost 
variance  data.  Cost  variances  for  programs  patterned  a  left  skewed  distribution.  The 
addition  of  more  data  points  did  not  normalize  the  distribution.  This  intuitively  makes 
sense,  if  one  considers  there  is  little  incentive  for  underruns  in  a  program. 

In  determining  whether  the  cost  variances  of  the  A- 12  were  exceptional  it  is 
important  to  determine  the  CBB  adjustments  for  other  programs.  Hypotheses  four  and 
five  tested  whether  certain  contracts  or  programs  experienced  greater  adjustments  to  the 
budget  than  other  programs. 
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D.  LITERATURE  REVIEW 

A  review  of  the  available  literature  revealed  no  directly  related  past  effort.  The 
most  widely  cited  source  in  analyzing  the  failure  of  the  A- 12  program  is  the  "Beach 
Report"  [Ref  2], 

The  Beach  Report  examined  issues  that  contributed  to  the  failure  of  the  A- 12 
program.  Specifically,  the  Beach  Report  sought  answers  to  three  questions,  "Did  the 
Navy... have  reason  to  anticipate  substantial  additional  cost  increase...  at  the  time  of  the 
Major  Aircraft  Review?  If  not,  why  not?  If  so,  were  senior  DON  and  DoD  leaders 
sufficiently  apprised  in  the  course  of  the  Review?"  [Ref.  2:  p.  2]  The  report  concluded 
that  sufficient  cost  performance  data  existed  to  indicate  that  the  contract  would  have 
significant  overruns  and  the  Program  Manager  "...erred  in  judgment  by  failing  to  anticipate 
substantial  additional  cost  increase  beyond  the  ceiling  of  the  FSD  contract..."  [Ref.  2:  p. 
29], 

Previous  studies  have  examined  the  affects  of  cost  and  schedule  overruns  on 
contract  costs  at  completion.  Dr.  David  S.  Christensen,  an  Associate  Professor  of 
Accounting  at  The  Air  Force  Institute  of  Technology,  examined  64  contracts  of  various 
types  and  programs  and  the  ability  of  a  program  to  recover  from  cost  overruns  at  various 
stages  [Ref.  3],  Using  a  linear  regression  model,  Christensen  tested  the  hypothesis  that 
contracts  were  unlikely  to  recover  from  cost  overruns.  His  results  indicated  that  programs 
experiencing  cost  overruns,  at  a  stage  of  completion  between  10  to  70  percent,  were 
highly  unlikely  to  finish  within  the  programmed  cost  baseline.  Furthermore,  he  found  that 
the  type  of  contract  used,  the  Service  managing  the  program  and  the  type  of  weapon  being 
acquired  had  no  significant  bearing  on  the  cost  overrun  incurred  in  the  program.  [Ref.  3] 

Christensen's  work  evolved  from  observations  made  by  Mr.  Gary  Christie  and  Mr. 
Wayne  Abba,  senior  program  analysts  in  the  OUSD  (A&T).  Abba  and  Christie  examined 
data  extracted  from  the  DAES  database  on  500  contracts  and  concluded  there  is  very  little 
chance  of  recovering  from  a  cost  overrun  once  a  program  is  more  than  15%  complete. 
"Given  a  contract  is  more  than  15  percent  complete,  the  overrun  at  completion  will  not  be 
less  than  the  overrun  to  date,  and  the  percent  overrun  at  completion  will  be  greater  than 
the  percent  overrun  to  date."  [Ref.  3] 
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E.  REVIEW  OF  PRIOR  RESEARCH 


Two  master's  theses  from  the  Air  Force  Institute  of  Technology,  Captain  Scott  R. 
Heise,  A  Review  of  Cost  Performance  Index  Stability  [Ref.  1 7],  and  Captain  Brian  D. 
Wilson,  An  Analysis  of  Contract  Cost  Overruns  and  Their  Impacts  [Ref.  30],  examined 
cost  performance  data  from  contracts. 

Captain  Heise's  research  examined  at  what  point  the  Cumulative  Cost  Performance 
Index  (CPIcum)  is  considered  to  be  stable.  Data  from  the  Office  of  the  Under  Secretary 
of  Defense  (Acquisition  &  Technology)  indicate  that  the  CPI  is  normally  stable  at  the  50 
percent  program  completion  point,  and  the  cumulative  CPI  does  not  improve  between 
program  completion  points  of  15  percent  to  85  percent.  Captain  Heise's  hypothesis  was 
that  the  CPI  is  stable  when  a  program  is  greater  than  50  percent  complete.  His  definition 
of  stability  was  that  the  CPI  would  not  vary  by  more  than  plus  or  minus  10  percent. 

Using  data  extracted  from  the  DAES  database,  Heise  examined  155  contracts. 

The  contracts  included  various  acquisition  phases  and  contract  types.  His  conclusion  was 
the  cumulative  CPI  is  stable  at  the  50  percent  point  and  will  stabilize  as  early  as  the  20 
percent  completion  point  if  the  program  maintains  a  stable  baseline.  Additionally,  Captain 
Heise  found  that  the  cumulative  CPI  tends  to  worsen  to  some  degree  as  the  program 
continues,  but  not  beyond  the  bounds  of  what  is  considered  a  stable  CPI.  His  conclusions 
confirm  previous  work  by  Abba  and  Christie,  that  there  is  very  little  chance  of  recovery 
from  a  program  cost  overrun  between  the  15  percent  and  85  percent  completion  points. 
[Ref.  17] 

Captain  Brian  D.  Wilson's  research  investigated  Christie's  assertion  that  of  500 
contracts  examined  since  1977,  the  cost  overrun  at  completion  would  be  higher  than  the 
cost  overrun  to  date.  Captain  Wilson  examined  65  contracts,  all  of  which  had  experienced 
cost  overruns.  His  findings  confirmed  the  claim.  He  also  found  that  the  cost  overruns 
tend  to  increase  as  the  program  progresses.  Captain  Wilson's  statistical  research  was 
performed  at  an  85  percent  confidence  level.  [Ref  30] 

Two  master's  theses  from  the  Naval  Postgraduate  School  were  used  for 
background  information  on  the  application  of  the  Rayleigh  Distribution  to  developmental 
programs.  Abemethy  [Ref.  1],  investigated  the  applicability  of  the  Rayleigh  Distribution 
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to  programs  and  Elrod  [Ref.  11],  tested  the  ability  of  the  Rayleigh  Distribution  to  provide 
estimates  at  completion. 

Abernethy's  work  examined  completed  contracts  to  test  the  ability  of  the  Rayleigh 
Distribution  to  model  expenditures.  The  results  showed  that  the  expenditures  of  most 
contracts  were  modeled  by  the  Rayleigh  Distribution,  but  the  predictive  capability  of  the 
model  was  insufficient  for  use  in  obtaining  accurate  cost  at  completion  estimates.  [Ref.  1] 

Elrod  tested  the  predictive  capability  of  the  Rayleigh  Distribution  using  two 
methods  to  determine  which  method  provided  the  best  predictions  of  final  cost.  Her 
sample  included  aircraft  programs  that  had  been  completed.  Elrod's  results  showed  that 
either  of  the  two  models  could  be  used  to  predict  final  costs.  The  model  used  in  this 
research  approximates  the  Putnam  model  examined  by  Elrod.  [Ref.  11] 

Two  unpublished  research  papers  were  relied  upon  to  provide  an  understanding  of 
the  application  of  the  Rayleigh  model.  The  first  paper.  Determining  a  Budget  Profile  from 
a  R  &  D  Cost  Estimate,  by  David  A.  Lee  et.  al  [Ref  18],  provided  verification  that  the 
Rayleigh  model  patterned  the  expenditures  in  a  developmental  program  as  well  as  a  simple 
description  of  the  model.  This  paper  is  available  in  the  Defense  Technical  Information 
Center  (DTIC)  database. 

The  second  paper,  Final-Cost  Estimates  for  Research  &  Development  Programs 
Conditioned  on  Realized  Costs,  by  Mark  A.  Gallagher  and  David  A.  Lee  [Ref.  14], 
provided  a  software  tool  that  was  used  to  develop  estimates  at  completion  for  the  A- 12 
program  and  cumulative  probabilities  of  occurrence.  This  paper  is  also  available  in  the 
DTIC  database,  and  was  presented  at  the  1995  Military  Operations  Research  Society 
Symposium. 

F.  SUMMARY 

This  Chapter  presented  the  objectives  of  the  research  and  the  hypotheses  to  be 
tested.  Most  programs  experience  cost  overruns.  Were  the  cost  overruns  in  the  A- 12 
Program  any  different  than  other  programs? 

The  following  chapters  will  provide  the  foundation  to  answer  the  question. 
Chapters  II  and  HI  will  present  the  general  background  information  on  the  Acquisition 
Process  and  the  A- 12  Program.  Chapter  IV  contains  the  data  from  the  58  contracts  drawn 
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from  the  DAES  database.  Chapter  V  is  the  analysis  of  the  data  and  the  determination  of 
where  the  A- 12  Program  was  at  versus  other  major  acquisitions.  Chapter  VI  reviews  the 
research  questions  and  provides  the  results  of  the  tests  of  the  hypotheses. 
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H.  BACKGROUND 


A.  OVERVIEW 

This  chapter  provides  an  overview  of  the  acquisition  process  as  it  is  currently 
structured,  and  a  summary  of  the  Cost/Schedule  Control  Systems  Criteria.  This  chapter 
also  introduces  techniques  for  estimating  costs  at  completion  for  an  EMD  program. 

The  intent  of  this  chapter  is  to  provide  the  reader  with  sufficient  background  in  the 
current  Defense  acquisition  process,  and  cost  estimating  procedures,  to  better  understand 
the  events  leading  up  to  the  termination  of  the  A- 12.  The  Acquisition  Process  information 
is  drawn  from  OMB  Circular  A- 109,  and  the  DoD  5000  Series. 

B.  THE  ACQUISITION  PROCESS 

1.  The  DoD  5000  Series 

The  DoD  5000  series  is  a  set  of  directives  and  instructions  issued  in  1991.  The 
series  consists  of  DoD  Directive  5000. 1,  Defense  Acquisition  [Ref.  6];  DoD  Instruction 
5000.2,  Defense  Acquisition  Management  Policies  and  Procedures  [Ref.  7];  and  DoD 
Manual  5000.2-M,  Defense  Acquisition  Management  Documents  and  Reports.  These 
replaced  the  1987  versions  of  DoDD  5000. 1  and  DoDI  5000.2. 

All  the  military  departments  are  subject  to  the  guidance  provided  in  the  5000 
series  which  provide  a  single  acquisition  system  for  all  defense  acquisition  programs.  The 
5000  series  is  implemented  in  a  phased  process  with  five  major  milestones,  as  shown  in 
Figure  1.  [Ref.  6] 


Figure  1 .  The  Acquisition  Process.  [Ref.  6] 
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2.  The  Mission  Need 

The  acquisition  process  begins  with  the  identification  of  a  deficiency  in  current 
tactics  or  equipment  that  can  only  be  satisfied  with  the  procurement  of  equipment.  DoDD 
5000. 1  requires  that  a  new  acquisition  be  initiated  only  after  "fully  examining  alternative 
ways  of  satisfying  identified  military  needs."  The  priority  in  examining  alternatives  as 
specified  in  DoDD  5000.1  is:  [Ref.  6] 

1.  Use  or  modification  of  an  existing  U  S.  military  system. 

2.  Use  or  modification  of  an  existing  commercially  developed  or  Allied 

system  that  fosters  a  non-developmental  acquisition  strategy. 

3.  A  cooperative  research  and  development  program  with  one  or  more 

Allied  nations. 

4.  A  new  Joint-Service  development  program. 

5.  A  new  Service-unique  development  program. 

The  increasing  sophistication  of  threat  air  defense  systems  degraded  the  capability 
and  survivability  of  the  A-6.  The  concept  exploration  for  the  A- 12  began  at  the  end  of  the 
cold  war  and  was  focused  on  penetrating  Soviet-type  air  defense  systems  using  low 
observable  (stealth)  technology.  The  United  States  Air  Force  (USAF)  systems  that 
incorporated  stealth  technology  were  not  suitable  for  carrier  operations  without  significant 
structural  modification  that  would  result  in  a  new  aircraft  from  what  was  originally 
designed  for  USAF  missions. 

The  initiation  of  the  A- 12  program  was  based  on  projected  USAF  purchases  of  the 
aircraft  in  larger  quantities  than  the  Navy.  Although  this  would  not  be  a  true  "joint" 
program,  the  A-12  procurement  was  based  on  two  Service's  needs. 

The  Services  submit  the  Mission  Need  Statement  (MNS)  to  a  Joint  Requirements 
Oversight  Council  (JROC).  The  JROC  determines  if  the  need  can  be  satisfied  by  other 
means.  The  JROC  determines  the  validity  of  the  need  and  forwards  the  MNS  to  the 
OUSD  (A&T)  for  approval  or  disapproval. 

The  A-12  Program  began  in  1984  with  approval  of  Milestone  0,  approval  for 
concept  studies.  The  Navy  contracts  for  Concept  Exploration  were  awarded  to  the 
contractor  teams  of  McDonnell  Douglas/General  Dynamics  and  Northrop/Grumman. 
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3.  Phase  0 

Phase  0  is  Concept  Exploration  and  Definition.  During  this  phase  the  priority  is  to 
develop  and  evaluate  various  alternatives  that  could  satisfy  requirements.  In  Phase  0 
initial  cost  estimates  are  conducted  to  determine  the  affordability  of  the  proposed  systems. 
Trade-offs  are  considered  between  cost  and  performance.  The  contract  for  Phase  0  in  the 
A- 12  program  was  issued  in  1984.  Phase  0  of  the  acquisition  process  begins  with 
Milestone  0,  and  approval  of  the  Mission  Need  Statement  (MNS). 

DoDI  5000.2  requires  that  during  this  phase  plans  must  be  made  for  competitive 
prototyping  unless  the  OUSD  (A&T)  decides  that  it  is  not  practical.  Plans  must  also  be 
made  to  allow  for  competitive  development  and  production. 

4.  Phase  I 

Milestone  I  is  the  start  of  a  new  acquisition  program  for  ACAT  ID  programs. 
ACAT  ID  programs  are  defined  as  programs  that  will  expend  more  than  $200  million  in 
1980  constant  dollars  in  RDT&E  or  the  expenditure  for  procurement  of  more  than  $1 
billion  in  1980  constant  dollars.  The  decision  authority  to  start  a  new  program  is  the 
OUSD  (A&T).  The  most  promising  designs  from  Phase  0  are  carried  forward  into  Phase 
I,  the  Demonstration  and  Validation  Phase. 

The  Defense  Acquisition  Board  (DAB)  reviews  the  program  at  each  milestone  to 
determine  if  it  meets  the  requirements  to  proceed  to  the  next  milestone.  If  a  new  program 
is  required,  the  parameters  for  the  development  of  the  system  are  established  and 
approved. 

The  threat  is  re-evaluated  at  Milestone  I  to  ensure  the  requirements  for  the 
program  still  exist  and  have  not  changed.  Total  costs  of  the  program  are  refined  and 
affordability  issues  are  considered.  The  intent  of  the  Milestone  decision  is  to  ensure  that  a 
new  program  is  not  started  when  other  means  can  satisfy  the  need. 

The  A- 12  Program  entered  Phase  I  in  1986,  with  the  award  of  the  Demonstration 
and  Validation  contract  to  McDonnell  Douglas/General  Dynamics  and 
Northrop/Grumman;  the  same  teams  that  had  conducted  the  concept  exploration  studies. 
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Phase  I  continues  to  define  the  critical  design  characteristics  and  projected 
capabilities  of  the  system.  Projected  costs  of  the  system  are  further  defined  as  the 
program  proceeds  through  the  Demonstration  and  Validation  Phase. 

Risk  is  considered  throughout  the  phases,  but  at  this  point  the  degree  of  risk  of 
incorporating  new  technologies  should  be  carefully  considered.  At  the  conclusion  of 
Phase  I  the  design  to  enter  the  Engineering  and  Manufacturing  Development  Phase  is 
selected,  and  the  design  concepts  should  be  relatively  firm. 

5.  Phase  11 

Milestone  II  is  the  approval  Milestone  by  the  DAB  to  enter  the  Engineering  and 
Manufacturing  Development  phase  of  a  new  system.  Of  crucial  importance  at  the 
Milestone  II  review  is  the  assessment  that  "the  technologies  and  processes  critical  to 
success  are  attainable."  The  DAB  is  additionally  charged  with  a  "rigorous"  assessment  of 
the  costs  of  the  system.  Prior  to  approval  to  enter  EMD  it  should  be  demonstrated 
through  testing  that  the  system  meets  contract  specification  requirements,  and  operational 
requirements. 

The  focus  during  EMD  is  on  the  producibility  of  the  system.  The  A- 12  contract  for 
EMD  was  awarded  to  the  team  of  McDonnell  Douglas  and  General  Dynamics  in  January 
1988,  four  years  after  initial  concept  studies  had  begun.  The  contractor  team  proposed  an 
aggressive  development  schedule  that  would  result  in  first  flight  thirty  months  after 
contract  award. 

6.  Phase  III 

Phase  III  is  the  production  of  the  actual  system.  Milestone  III  is  a  critical 
milestone  in  a  program's  lifecycle.  Milestone  III  is  the  Production  Approval  for  the 
system.  The  objectives  as  defined  in  DoD  5000.2  are  to:  (1)  determine  if  the  results  of 
Phase  II  warrant  continuation,  and  (2)  establish  a  Production  Baseline  containing  refined 
program  cost,  schedule  and  performance  objectives.  [Ref.  7]  Phase  III  is  normally  divided 
into  two  unofficial  sub-phases:  Low  Rate  Initial  Production  (LRIP)  and  Full  Rate 
Production.  The  purpose  of  LRIP  is  to  reduce  the  risk  of  producing  the  system  by 
producing  it  in  small  quantities  initially,  then  once  the  producibility  is  verified,  to  begin 
Full  Rate  Production. 
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7.  Phase  IV 

Phase  IV  usually  completes  the  acquisition  lifecycle.  Phase  IV  is  initiated  by 
Milestone  IV,  Major  Modification  Approval.  The  purpose  of  the  decision  at  Milestone  IV 
is  to  determine  if  a  major  modification  is  required.  In  addition  to  major  modifications. 
Phase  IV  includes  the  operations  and  support  of  the  fielded  equipment. 

C.  COST  AND  SCHEDULE  CONTROL  SYSTEMS  CRITERIA 

1.  Overview 

Throughout  the  acquisition  process  the  cost  of  the  proposed  system  is  of  great 
importance.  Cost  must  be  considered  at  each  milestone  decision.  All  ACAT  ID  programs 
must  incorporate  C/S  CSC  into  the  management  of  the  program.  Cost  and  schedule 
control  systems  allow  program  managers  to  track  expenditures  of  resources  in  the 
program. 

2.  Requirements 

DoD  5000.2  requires  that  the  contractor  submit  C/S  CSC  information  on  all 
significant  contracts  that  are  not  firm  fixed-price.  The  definition  of  a  significant  contract 
for  the  requirement  is  a  contract  with  Research,  Development,  Test  and  Evaluation 
(RDT&E)  values  of  $60  million  or  more,  and  procurement  contracts  valued  at  $250 
million  or  more,  in  fiscal  year  1990  constant  dollars.  Contracts  that  are  not  determined  to 
be  significant  must  submit  the  Cost/Schedule  Status  Report  (C/SSR).  [Ref.  7] 

According  to  DoD  5000.2,  the  purposes  of  C/S  CSC  requirements  are  to:  (1) 
provide  uniform  evaluation  criteria  to  ensure  contractor  cost  and  schedule  management 
systems  are  adequate,  (2)  to  provide  an  adequate  basis  for  responsible  decision  making  by 
contractor  management  and  DoD  personnel,  and  (3)  to  bring  to  the  attention  of  DoD 
contractors,  and  encourage  them  to  accept  and  install,  management  control  systems  and 
procedures  that  are  most  effective  in  meeting  requirements  and  controlling  contract 
performance.  [Ref.  7] 

Contractors  report  C/S  CSC  data  monthly.  DoD  5000.2  requires  that  3  elements 
must  be  reported  .  (1)  comparison  of  budgeted  cost  for  work  scheduled  and  the  budgeted 
cost  of  work  performed;  (2)  comparison  of  the  budgeted  cost  of  work  performed  and  the 
actual  cost  of  work  performed;  (3)  variances  resulting  from  the  differences  in  comparisons 
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between  budgeted  cost  of  work  schedule  and  performed,  and  variances  between  budgeted 
cost  of  work  performed  and  actual  cost  of  work  performed.  [Ref.  7] 

Of  particular  interest  to  program  managers  and  DoD  oversight  personnel  are  the 
latest  revised  estimates  (LRE)  and  estimates  at  completion  (EAC)  that  are  reported.  The 
computation  of  these  estimates  may  vary  but  there  are  four  standard  techniques  that  are 
explained  later. 

3.  Purpose  of  C/S  CSC 

C/S  CSC  are  a  management  tool  for  monitoring  actual  progress  on  a  contract 
versus  the  baseline.  C/S  CSC  begins  with  the  formulation  of  a  work  breakdown  structure 
(WBS)  that  identifies  the  tasks  and  sub-tasks  that  constitute  the  complete  work  package 
of  the  contract.  The  WBS  identifies  which  particular  tasks  may  be  ahead  or  behind 
schedule  and  identifies  areas  where  the  reallocation  of  resources  is  required.  The  tasks 
and  sub-tasks  are  provided  parallel  work  package  budgets  that  are  traceable  to  the  work 
being  accomplished. 

There  are  several  methods  of  allocating  value.  One  method  is  when  work  package 
budgets  are  time  phased;  no  dollars  are  credited  to  a  work  package  not  started.  Once  a 
work  package  is  started,  50  percent  of  the  designated  budget  is  credited  to  the  package, 
while  the  remaining  50  percent  is  credited  at  completion.  An  alternative  means  of 
allocating  earned  value  is  to  assign  1 00  percent  of  the  value  of  the  work  package  only 
when  it  is  completed.  Some  distortion  will  be  experienced  in  the  actual  progress  of  the 
contract  using  either  of  these  methods. 

The  exact  accounting  system  that  a  contractor  must  use  to  maintain  C/S  CSC  data 
is  not  specified  by  the  Government.  A  system  that  provides  the  required  data  does  not 
need  to  be  modified  to  meet  contractual  requirements 

4.  C/S  CSC  Elements 

Contractors  submit  the  Cost  Performance  Report  (CPR),  which  includes  the  C/S 
CSC  data,  to  the  program  manager.  For  ACAT  ID  programs,  program  managers  forward 
CPR  information  with  the  Defense  Acquisition  Executive  Summary  (DAES)  Report  to  the 
OUSD  (A&T).  Within  the  OUSD  (A&T)  office  a  database  is  maintained  of  DAES  CPR 
information.  In  this  research,  the  following  elements  of  the  CPR  are  of  importance: 
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a.  Budgeted  Cost  of  Work  Scheduled  (BCWS) 

The  BCWS  is  the  dollar  amount  in  current  dollars  budgeted  to  accomplish 
a  specified  work  package  in  a  specified  time. 

b.  Budgeted  Cost  of  Work  Performed  (BCWP) 

The  BCWP  is  the  cumulative  dollar  amount  of  the  sum  of  the  completed 
work  and  the  open  work  packages.  The  BCWP  is  the  baseline  item  in  cost  reports.  It 
represents  what  portion  of  the  work  has  been  accomplished  in  dollars  [Ref.  30],  As  used 
in  this  research,  the  BCWP  is  the  cumulative  dollar  amount  of  the  projected  costs  of  the 
work  scheduled  at  a  specified  completion  point.  BCWP  information  is  drawn  from  the 
DAES  database. 

c.  Actual  Cost  of  Work  Performed  (ACWP) 

ACWP  is  the  actual  costs  incurred  and  recorded  in  accomplishing  the  work 
performed  within  a  specified  time  period  [Ref.  6],  As  used  in  this  research  ACWP  is  the 
cumulative  dollar  amount  of  the  actual  costs  for  the  work  accomplished  at  a  percentage 
completion  point  of  the  program.  ACWP  information  is  drawn  from  the  DAES  database. 

d.  Budget  at  Completion  (BAC) 

The  BAC  is  the  dollar  amount  of  the  contract  in  terms  of  the  cumulative 
cost  of  the  BCWS.  It  is  the  contractually  specified  dollar  amount. 

e.  Contract  Budget  Base  (CBB) 

The  CBB  is  the  negotiated  contract  cost  plus  estimates  of  authorized  but 
unpriced  work.  The  CBB  includes  Management  Reserve,  which  is  money  reserved  for 
within-scope  changes  to  the  contract. 

f.  Estimate  at  Completion  (EAC) 

EAC  is  the  contractor's  estimate  of  the  cumulative  cost  at  completion  of 
the  sum  of  all  work  packages.  EAC  in  its  simplest  form  is  the  dollar  amount  of  all  costs 
incurred  to  date  plus  the  dollar  amount  of  all  costs  remaining  to  complete  the  contract. 

The  EAC  may  be  derived  by: 

EAC  =  ACWP  +  (BAC  -  BCWP)  (2.1) 
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g.  Cost  Variance  (CV) 

The  CV  is  the  difference  between  the  dollar  amount  that  was  planned  for 
the  work  package  and  the  actual  cost  for  the  accomplished  work  package.  It  is 
determined  by: 

CV  =  BCWP  -  ACWP  (2.2) 

h.  Percent  Cost  Variance 

The  percent  cost  variance  is  a  convenient  means  of  comparing  the 
magnitudes  of  differences  between  budgeted  costs  and  actual  costs.  This  research  uses  it 
as  a  comparison  between  different  programs  of  varying  budgets  and  schedules,  It  is  a 
useful  analysis  tool  and  is  given  by: 

%  CV  =  CV/BCWP  (2.3) 

i.  Cost  Performance  Index,  Efficiency  (CPI  (E)) 

The  CPI  (E)  measures  the  efficiency  with  which  work  has  been 
accomplished  by  comparing  the  ratio  of  the  budgeted  cost  of  work  performed  to  the  actual 
cost.  It  is  given  by: 

CPI  (E)  =  BCWP/ACWP  (2.4) 

j.  Cost  Performance  Index,  Performance  (CPI  (P)) 

The  CPI  (P)  is  simply  the  inverse  of  the  CPI  (E).  This  measures  the  actual 
cost  of  each  planned  dollar  of  work  accomplished.  It  is  given  by: 

CPI  (P)  =  ACWP/BCWP  (2.5) 

5.  Estimates  at  Completion 

The  estimate  at  completion  (EAC)  can  be  derived  a  number  of  ways.  Some  of  the 
more  common  methods  involve  using  an  efficiency  index  of  past  work  performed  and 
extrapolating  this  into  the  future  to  determine  the  final  cost  of  the  project.  The  Cost 
Performance  Index  (CPI)  is  a  commonly  used  tool  to  estimate  the  final  cost.  Three 
methods  for  obtaining  CPI  estimates  for  completion  costs  are: 

a.  Cumulative  CPI  (CPIcum) 

The  EAC  (CPIcum)  is  given  by: 

EAC  (CPIcum)  =  BAC/CPI(E)  (2.6) 
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This  EAC  uses  the  cumulative  budgeted  costs  to  date  and  the  cumulative  actual 
costs  to  date  to  derive  an  efficiency  of  work  index.  The  assumption  is  the  efficiency  index 
derived  will  remain  stable  into  the  future.  Gary  Christie  and  Wayne  Abba  examined  over 
500  contracts  from  the  DAES  database  and  found  that  the  Cumulative  CPI  will  remain 
stable  between  15  and  85  percent  complete  in  a  program.  Christie  and  Abba's 
observations  were  verified  empirically  by  Heise  [Ref.  17],  Heise  examined  a  sample  of 
155  contracts  from  the  DAES  database  and  confirmed  that  the  Cumulative  CPI  remains 
stable  between  20  percent  complete  and  85  percent  complete. 

b.  Weighted  CPI 

This  method  of  estimating  the  final  cost  of  a  contract  is  essentially  the  same 
formula  as  (2.6)  above.  The  primary  difference  is  the  use  of  more  recent  data  in  the  CPI 
(E)  divisor.  Commonly  used  weightings  include  3  and  6  month  CPI  measurements.  It  has 
been  found  that  the  Weighted  CPI  will  tend  to  give  a  higher  estimate  at  completion  than 
the  Cumulative  CPI.  Normally,  a  range  of  estimates  will  be  given  to  the  program  manager 
by  the  cost  analyst. 

c.  CPI/SPI 

The  CPI/SPI  estimate  at  completion  is  given  by: 

EAC  (CPI/SPI)  =  BAC  x  [(CPI(P)/SPI(E)]  (2.7) 

The  CPI/SPI  estimate  is  based  on  a  ratio  of  the  cost  of  the  work  performed  and  the 
scheduled  time  of  the  work  performed.  This  approach  balances  the  cost  and  schedule  to 
determine  the  ending  cost. 

6.  Cost  Overruns 

Cost  overruns  are  common  in  weapon  system  development  programs.  A  Rand 
Report  [Ref.  10]  found  the  average  cost  growth  in  the  development  phase  was  25  percent, 
and  the  average  cost  growth  in  the  production  phase  was  18  percent.  The  report  also 
found  that  overruns  varied  by  Service.  No  single  cause  for  the  cost  growth  could  be 
determined.  Two  variables  that  showed  a  strong  correlation  to  cost  growth  were  program 
size  and  program  maturity.  The  smaller  the  program  the  greater  the  cost  variance  from 
the  baseline,  and  the  longer  the  program,  the  greater  the  cost  variance.  Table  1  provides  a 
summary  of  the  Rand  reports  findings. 
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Program  Type 

Cost 

Summary 

Cost 

Growth 

Growth 

Vehicles 

1.77 

Total  RDT&E 

1.25 

Aircraft 

1.28 

Procurement 

1.18 

Electronics 

1.24 

Munitions 

1.22 

Army  Programs 

1.35 

Missiles 

1.17 

USAF  Programs 

1.20 

Space  Systems 

1.16 

Navy  Programs 

1.16 

Helicopters 

1.13 

Table  1.  Cost  Growth  by  Type  of  Program.  [Ref.  10] 


Surprisingly,  the  Rand  report  found  that  vehicles  experience  the  greatest  cost 
growth.  The  Rand  study  adjusted  the  budgets  for  inflation.  The  report  did  not  test  for 
statistical  difference  between  either  the  types  of  programs  or  Service.  Measuring  cost 
variances  is  a  relatively  straightforward  procedure.  Estimating  costs  at  completion  is  more 
difficult.  The  next  section  introduces  a  method  that  can  provide  estimates  of  the 
completion  costs  of  a  EMD  program. 

D.  THE  RAYLEIGH  DISTRIBUTION  METHOD 

1.  Overview 

The  accuracy  of  a  cost  estimate  is  critical  to  the  use  of  any  method  as  a  tool. 
Expenditures  in  developmental  programs  tend  to  follow  a  distinctive  curve,  often  referred 
to  as  the  Rayleigh  Curve.  This  curve  can  be  used  to  model  natural  events  such  as  the 
growth  of  biological  systems.  Developmental  projects  tend  to  pattern  this  same  type  of 
growth  during  the  lifecycle  of  the  project.  There  are  natural  build-ups  and  build-downs  in 
the  level  of  resources  consumed. 

Norden  used  this  curve  in  Useful  Tools  for  Project  Management,  to  model  the 
costs  and  resource  use  of  software  development.  Abemethy  found  the  curve  could 
accurately  model  the  cost  expenditure  pattern  of  weapon  system  programs.  Elrod 
examined  the  application  of  the  curve  in  EMD  programs  using  two  different  models  and 
found  that  it  fit.  Lee,  Hogue  and  Gallagher  developed  a  technique  using  the  Rayleigh 
Function  to  estimate  Research  and  Development  budget  profiles.  Gallagher  and  Lee 
developed  a  technique  based  on  the  Rayleigh  model  to  provide  probabilities  for  the 
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estimate  at  completion.  Substantial  prior  work  has  been  accomplished  that  indicates  the 
Rayleigh  Function  models  actual  expenditures  well. 

Here  the  Rayleigh  model  is  used  to  estimate  the  ending  costs  of  the  A- 12  program, 
had  it  continued  to  completion  in  EMD.  Figure  2  provides  a  graphical  representation  of 
the  cumulative  expenditure  curve  that  models  developmental  program  expenditures. 


%  Time  Elapsed 


Figure  2.  Typical  "S"  Shaped  Rayleigh  Curve.  [Ref.  18] 

2.  Formulae 

The  general  form  of  the  Rayleigh  cumulative  density  function  is  given  by: 

F(t)  =  l-exp(-at2)  (2.8) 

Where  a  is  a  time-scale  parameter  and  t  is  the  time  period.  [Ref.  18]  Equation 

(2.8)  is  adapted  for  use  in  developmental  programs  in  the  following  function:  [Ref.  18] 

V(t)  =  d[l-(exp(-at2)]  (2.9) 

Where  V(t)  is  the  earned  value  at  time  t.  d  is  a  positive  constant  that  is  equal  to 

the  total  budget  of  the  project  (D)  divided  by  .97.  [Ref.  18] 

D  =  V(tf)  =  .97d  (2.10) 

a  is  a  constant  that  for  EMD  projects  has  been  estimated  to  equal: 

a  =  3.5/t2f  (2.11) 

With  tf  equal  to  the  final  time  of  the  project.  [Ref.  18] 

Often  the  final  time  of  the  project  is  not  known  with  certainty.  If  the  time  of  peak 

rate  of  expenditures  is  known  or  estimated,  then  a  can  be  estimated  by: 

a  =  l/(2t2p)  (2.12) 
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Where  tp  is  equal  to  the  time  of  peak  rate  of  expenditures.  The  time  of  peak  rate 
of  expenditures  assumes  added  importance  for  programs  such  as  aircraft  or  missiles  which 
have  scheduled  first  flight  dates.  It  has  been  found  that  the  peak  rate  of  expenditure  often 
occurs  at  the  time  of  first  flight.  By  taking  the  first  derivative  of  equation  (2.9)  and  setting 

it  equal  to  zero,  the  time  of  peak  expenditure  rate,  or  first  flight  can  be  found.  [Ref.  18] 

V'(t)  =  2at[exp(-at2)]  (2.13) 

Algebraic  manipulation  of  equation  (2.9)  yields  a  formula  for  estimating  the  costs 

at  completion,  given  a  current  ACWP  at  a  specified  time:  [Ref.  18] 

d  =[  V(t)/(l-exp(-at2 ))]  (2.14) 

E.  APPLICATION  OF  THE  RAYLEIGH  FUNCTION  TO  THE  A-12  PROGRAM 

1.  Overview 

Early  in  this  research  the  author  realized  that  an  appropriate  tool  for  estimating 
costs  at  completion  was  needed.  While  common  methods  for  estimating  costs  at 
completion  involve  the  use  of  efficiency  indices,  this  method  was  found  to  be 
inappropriate  for  estimations  of  completion  costs  when  the  CBB  is  subject  to  future 
changes.  The  CPI  technique  is  based  on  the  current  budget  and  its  estimates  will  be  biased 
in  this  regard.  The  Rayleigh  technique  provides  an  estimate  for  what  the  future 
completion  cost  will  be  independent  of  current  budgets. 

2.  Justification 

Abemethy's  previous  work  demonstrated  a  high  correlation  in  the  model's  ability 
to  provide  ACWP  estimates,  given  the  final  cost.  Abemethy,  concluded  that  while  the 
Rayleigh  Function  could  effectively  model  ACWP,  given  final  actual  costs,  it  was  not  a 
good  predictor  of  final  costs  [Ref.  2],  The  question  then  becomes  is  the  Rayleigh  method 
a  better  predictor  of  final  cost  than  the  CPI  methods? 

The  Rayleigh  Function  was  applied  to  a  sample  of  10  contracts  using  equation 
(2. 14)  to  determine  if  the  estimates  at  completion  were  closer  than  estimates  using 
efficiency  indices.  An  estimate  at  completion  for  each  percentage  completion  point  was 
computed  using  the  Rayleigh  Function,  the  Cumulative  CPI,  and  a  Weighted  CPI.  The 
total  variance  from  the  actual  estimate  at  completion  and  the  Rayleigh  estimate  showed  an 
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absolute  difference  much  less  than  the  two  other  methods,  indicating  a  more  accurate 
model. 

Further  refining  the  model  by  iteratively  adjusting  the  shape  parameters  and 
establishing  time  bracket  parameters  resulted  in  even  less  variance  in  the  Rayleigh 
estimates.  The  Rayleigh  technique  was  able  to  achieve  results  that  were  300  percent 
better  than  the  Weighted  CPI  method  and  200  percent  better  than  the  Cumulative  CPI 
method.  Elrod  varied  the  alpha  value  of  the  model  to  determine  a  best  fit  to  the  data.  In 
this  research  the  alpha  value  is  fixed  at  .00035  and  the  power  of  t  from  equation  (2. 14)  is 
allowed  to  vary.  The  alpha  value  is  able  to  be  fixed  at  .00035  by  standardizing  all 

contracts  to  the  same  time  period.  This  was  accomplished  by  using  percent  completion 
points  instead  of  months.  Therefore  every  contract's  time  period  consists  of  100  and  a 
remains  constant  at  .00035. 

To  optimize  the  model  to  the  data  the  alpha  value  can  not  be  a  constant.  This  was 
adjusted  for  by  allowing  the  constant  power  of  2  for  the  power  of  (-at)  to  vary.  In  effect 
equation  (2. 14)  becomes: 

d  =  [V(t)/(l-exp(-af))]  (2.15) 

Equation  (2.15)  was  used  to  estimate  the  final  cost  of  the  A- 1 2  Program.  This 

equation  varies  from  the  original  Rayleigh  function  by  the  addition  of  the  variable  x.  The 
basis  of  the  estimating  technique  remains  equivelent  to  the  Rayleigh  methodology. 

The  constant  power  2  was  replaced  by  the  variable  x  to  allow  the  model  to  adjust 
based  on  the  percentage  completion  the  program  had  achieved.  During  the  research 
hundreds  of  tests  were  conducted  using  the  Rayleigh  model  and  actual  contract  data.  It 

was  found  that  the  Rayleigh  model  consistently  overestimated  the  completion  costs  early 
in  the  program  and  late  in  the  program.  By  fixing  a  and  allowing  the  power  oft  to  vary 
according  to  the  percentage  complete  a  significantly  more  accurate  model  was  obtained. 

In  the  research  the  variable  x  was  allowed  to  assume  one  of  ten  values  corresponding  to 
percentage  complete  brackets.  The  addition  of  this  third  variable  to  the  function  changes 
the  essential  characteristics  of  what  is  considered  the  Rayleigh  model.  A  full  examination 
of  the  effects  of  this  addition  are  outside  the  scope  of  this  research. 

Gallagher  refined  the  Rayleigh  model  using  Multiple  Model  Adaptive  Estimation 
(MMAE)  [Ref.  14]  to  provide  Bayesian  statistical  probabilities  of  outcomes  for  costs  at 
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completion.  By  refining  the  model  in  this  method  it  provided  cumulative  probabilities  of 
the  Rayleigh  produced  completion  costs.  Gallagher's  MMAE  model  is  used  in  this 
research  to  provide  cumulative  probabilities  of  occurrence  of  the  EAC  for  the  A- 12 
program. 

F.  THE  BETA  DISTRIBUTION 

Information  on  cost  variances  was  collected  from  58  developmental  contracts. 
Based  on  the  sample  and  the  hypothesis  to  be  tested,  data  from  all  58  contracts  were  not 
used.  The  requirements  that  the  data  had  to  meet  for  each  research  question  are  contained 
in  Chapter  IV,  Data. 


The  sample  data  did  not  approximate  a  normal  distribution.  In  this  research  a  cost 
overrun  is  considered  a  negative  cost  variance  while  a  cost  underrun  is  considered  a 
positive  cost  variance.  The  distribution  of  these  data  was  skewed  to  the  left,  with  the 
mean  close  to  zero  and  data  points  extending  far  into  the  left  tail. 

Using  Best  Fit  [Ref.  20],  the  distribution  that  was  most  representative  of  cost 

overruns  was  the  Beta  distribution.  The  Beta  distribution  can  assume  many  shapes  based 
on  the  two  parameters  (v,  w )  that  determine  its  shape.  Figure  3  is  provided  to  give  the 
reader  some  insight  into  the  degree  to  which  the  Beta  distribution  modeled  the  data.  More 

information  on  the  distribution  of  the  data  is  contained  in  Appendix  B. 


Figure  3.  The  Beta  Distribution. 
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G.  SUMMARY 

This  chapter  provided  an  overview  of  the  Acquisition  Process  and  the  C/S  CSC 
terminology  that  will  be  used  throughout  this  paper. 

The  technique  used  for  obtaining  EACs  for  the  A- 12  program  was  presented. 

While  there  are  endless  techniques  for  estimating  completion  costs,  the  Rayleigh  model  is 
used  based  on  prior  research  and  its  ability  to  estimate  costs  independent  of  current 
budgets. 

Cost  variances  are  usually  negative.  The  distribution  of  cost  variances  for  the 
sample  was  skewed  to  the  left.  The  Beta  distribution  tended  to  model  cost  overruns  better 
than  the  Normal  distribution.  A  graphical  presentation  of  the  Beta  distribution  is  helpful  in 
picturing  the  distribution  of  cost  overruns. 

Chapter  III  completes  the  background  portion  of  this  paper.  Chapter  III  examines 
the  A- 12  Program  and  the  cost  variances  that  caused  the  termination  of  the  Program. 


; 
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m.  THE  A-I2  PROGRAM 


A.  OVERVIEW 

This  chapter  is  intended  to  give  the  reader  background  information  on  the  Navy's 
A- 12  development  program.  As  shown  in  Chapter  II,  cost  overruns  in  defense 
acquisitions  are  not  unusual.  This  chapter  supports  the  primary  research  question  by 
examining  the  A- 12  Program  and  the  cost  overruns  that  occurred.  For  a  more  in-depth 
view  of  the  A- 12  Program  the  reader  is  referred  to  The  Beach  Report  [Ref.  2],  the  Navy's 
internal  review  of  the  Program  that  was  conducted  in  1990. 

B.  MISSION  NEED 

The  A-12  Avenger  was  designed  to  replace  the  A-6  Intruder  in  the  Navy's  carrier 
based  fleet.  Its  projected  survivability  and  adaptability  would  allow  the  Navy  to  also 
replace  the  EA-6B,  Advanced  Tactical  System  aircraft.  The  Navy  also  considered  the  air- 
to-air  capability  of  the  A-12  in  possibly  replacing  the  F-14s  in  the  fleet  defense  role.  The 
various  roles  that  the  A-12  could  fill  would  provide  a  more  streamlined  logistical  system 
by  reducing  the  various  spare  parts  that  would  be  carried  in  inventory  [Ref.  21],  The 
aircraft  was  to  be  a  carrier-based  medium  range  attack  aircraft  with  low  observable 
(stealth)  characteristics.  The  United  States  Air  Force  (USAF)  was  interested  in  procuring 
up  to  400  A-12  variants  that  would  not  require  the  structural  hardening  to  land  on  aircraft 
carriers.  The  Air  Force's  version  was  planned  to  be  the  replacement  for  the  FI  1 1 .  The  Air 
Force  planned  on  taking  initial  delivery  in  1995.  [Ref.  9] 

The  first  version  of  the  A-6  entered  the  fleet  in  1963  as  an  all  weather,  day/night 
medium  attack  aircraft.  The  latest  version  of  the  A-6,  the  A-6E,  was  introduced  in  1972 
and  procurement  continued  through  1987.  In  the  1980's  wing  cracks  began  appearing  in 
the  inventory  of  the  A-6s.  The  stress  of  landing  on  carriers  required  that  when  the  wings 
reached  67  percent  of  their  life,  the  plane  was  limited  to  restricted  duty  until  the  wings 
were  replaced.  By  1988,  most  of  the  inventory  of  A-6s  was  rapidly  degrading.  In  1988, 
the  year  the  contract  for  the  Full-Scale  Development  of  the  A-12  was  awarded,  the  Navy 
placed  its  last  production  order  for  the  A-6.  The  A-12,  with  an  Initial  Operational 
Capability  (IOC)  of  1994,  was  to  replace  the  A-6s.  The  first  squadron  was  projected  to  be 
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combat  ready  by  late  1996  [Ref.  21],  Figure  4  shows  the  projected  A- 12  aircraft 
superimposed  over  the  F-14  and  the  A-6. 


Figure  4.  Aircraft  Comparison.  [Ref.  21] 


C.  EARLY  DEVELOPMENTAL  WORK 

In  1984  Concept  Exploration  work  began  on  the  A- 12  Program.  The  F-l  17 
Stealth  Fighter  proved  the  validity  of  stealth  technology.  The  F-l  17  was  operational  for 
one  year  with  the  USAF  in  the  Nevada  Desert,  however,  it  was  still  tightly  classified.  The 
early  development  work  on  the  A- 12  was  also  classified  as  a  special  access  program. 

In  1984,  two  separate  contractor  teams  began  concept  exploration  for  the  new 
aircraft.  This  Program  was  initiated  in  1984  with  the  start  of  Concept  Exploration.  In 
June  1986  the  same  contractor  teams  were  awarded  contracts  to  continue  work  into 
Demonstration  and  Validation.  In  June  1986,  Captain  Lawrence  Elberfeld  was  appointed 
as  the  Program  Manager  of  the  Program  which  was  now  in  Phase  I  of  the  Acquisition 
Process.  Captain  Elberfeld  participated  in  the  source  selection  for  the  A- 12.  Phases  0  and 
I  of  the  acquisition  cost  $2.8  billion.  [Ref.  25] 

D.  FULL-SCALE  DEVELOPMENT 

1.  The  Contract 

The  FSD  contract  was  awarded  to  McDonnell  Douglas/General  Dynamics  in 
January  1988.  The  contract  was  a  Fixed-Price-Incentive-Firm  type  contract  with  a  share 
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ratio  of  60/40  above  contract  target  cost.  The  target  price  was  $4,379  billion,  the  target 
cost  was  $3,981  billion  and  the  ceiling  price  was  $4.77  billion  [Ref.  9],  On  any  overruns 
above  target  cost,  from  $4.4  billion  to  $4.8  billion,  the  Government  would  pay  60  cents  of 
every  dollar  and  the  contractor  would  pay  40  cents.  The  Point  of  Total  Assumption  was 
$4.65  billion.  Progress  payments  to  the  contractor  were  approved  at  the  80  percent  rate. 

The  contract  required  the  full-scale  development  and  prototype  production  of  the 
aircraft.  The  original  first  flight  was  scheduled  for  June  1990.  This  date  was  adjusted 
several  times  throughout  contract  performance,  until  just  prior  to  termination  the 
projected  first  flight  date  was  March  1991.  The  time  of  first  flight  in  an  aircraft 
development  contract  is  of  crucial  importance  in  estimating  cost  expenditures.  As  was 
explained  in  Chapter  II  in  more  detail,  the  point  of  first  flight  is  usually  the  point  of  peak 
expenditures  of  the  program.  Given  this  point  the  total  cost  of  the  contract  at  completion 
may  be  estimated.  Historical  data  indicate  that  40  percent  of  the  development  costs  are 
expended  at  the  point  of  first  flight  [Ref.  14].  The  impact  of  delaying  the  first  flight  results 
in  increased  cost  and  extended  development  schedule  [Ref.  18],  Table  2  provides  the 
adjustments  made  to  the  scheduled  first  flight  of  the  aircraft  over  the  performance  of  the 
EMD  contract. 


1st  FLIGHT 

DATE 

DATE  OF  SLIP 

SOURCE 

$7.  June  ;19gCk  •. . ^ 

Contractually  Specified 

September  IMG 

November  1989 

'Novi  89  DAESi  report  :  :  ^ 

December  1990 

Contractor  Brief  to  MAR 

Steering  Committee 

May  1990 

Contractor  Brief  to  PM  : : :« 

December  1991 

November  1990 

Contractor  request  to 
restructure 

December  1990 

CAIG  estimate  submitted 
to  the  DAB 

Table  2.  A- 12  First  Flight  Adjustments. 


The  contract  also  required  delivery  of  "eight  flight  test  aircraft  and  five  full-scale 
ground  test  articles..."  Additional  provisions  of  the  contract  provided  for  three  production 
option  lots  of  4,  6,  and  16  aircraft,  and  an  option  for  a  fourth  production  lot  at  the 
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completion  of  the  Critical  Design  Review.  The  first  two  production  lots  were  for  pilot 
production  and  the  third  and  fourth  lots  were  low  rate  initial  production.  [Ref.  9] 

2.  Costs 

Initial  cost  estimates  were  based  on  the  purchase  of  858  total  aircraft  at  an  annual 
procurement  rate  of  48  aircraft.  With  the  reduction  of  the  fleet  to  14  aircraft  carriers  the 
requirement  for  A- 12s  dropped  to  620  aircraft  purchased  at  an  annual  rate  of  36. 

The  cost  of  the  A- 12  was  initially  projected  to  be  $86.6  million  per  aircraft,  given  a 
production  rate  of  48  per  year.  When  the  requirement  dropped  to  36  aircraft  per  year,  the 
cost  per  aircraft  increased  to  over  $100  million  per  aircraft.  The  Lot  I  pilot  production 
was  priced  at  $1.2  billion  for  6  aircraft  [Ref.  24],  As  production  continued  and  the 
quantity  produced  increased  with  the  anticipated  USAF  demand,  prices  were  expected  to 
fall,  according  to  Learning  Curve  theory. 

A  graphical  cost  comparison  of  the  A- 12  to  other  combat  aircraft  is  shown  in 
Figure  5.  Costs  shown  are  adjusted  to  1995  dollars  using  Department  of  the  Navy 
inflators.  All  data  are  drawn  from  IJ.S.  Weapon  Systems  Costs  [Ref.  5], 


The  cost  data  presented  are  from  1975  to  1995,  and  adjusted  to  1995  constant 
dollars  using  Department  of  the  Navy  inflator  values.  As  shown,  the  cost  of  the  A- 12 
would  have  been  significantly  higher  than  other  aircraft  in  the  inventory.  The  cost  data  for 


30 


the  F-117  are  still  restricted,  so  no  comparison  can  be  made  to  other  stealth  fighter 
aircraft. 

3.  Costs  at  Completion 

The  decision  to  terminate  the  A- 12  Program  was  based  on  the  cost  overruns  that 
the  Program  was  experiencing  and  the  uncertain  estimate  of  costs  at  completion.  The 
Secretary  of  Defense  stated  that  the  inability  of  being  able  to  determine  the  final  cost  of 
the  Program  was  a  primary  reason  for  the  termination. 

The  cost  at  completion  is  driven  by  the  first  flight  date.  As  the  programmed  first 
flight  of  the  aircraft  is  extended,  total  program  time  is  extended  and  the  costs  increase. 
The  difficulty  of  determining  a  final  cost  is  highlighted  in  Table  3.  Table  3  provides  final 
cost  estimates  of  the  Program  determined  by  the  Department  of  Defense  Inspector 
General  (DoD  IG). 


IF  1ST  FLIGHT  IS: 

AND  %  COMPLETE  AT  1ST  FLIGHT  IS! 

40 

50 

60 

JUNE  91 

$14.28  B 

$11.42  B 

$9.52  B 

DEC  91 

15.86 

1269 

10.58 

MAR  92 

17.36 

13.88 

11.57 

Table  3.  DoD  IG  Estimates  at  Completion.  [Ref.  9] 


The  DoD  IG  estimates  were  prepared  by  the  Cost  Analysis  Improvement  Group 
(CAIG).  The  CAIG  estimates  provide  EACs  based  on  three  possible  first  flight  dates  and 
three  possible  percentage  completion  points.  The  June  1991  first  flight  date  is  the  most 
optimistic  date.  The  range  of  values  is  based  on  assumptions  of  the  percentage  of  the 
programs  that  would  be  complete  at  first  flight,  with  60  percent  being  the  most  optimistic. 

4.  The  Contractors 

The  solicitation  for  the  A- 12  required  a  teaming  arrangement.  A  teaming 
arrangement  is  intended  to  provide  two  qualified  contractors  for  production  to  reduce  unit 
costs.  When  the  FSD  phase  of  the  A-12  contract  began  10  U.S.C.  2438  required  the 
Department  of  Defense  to  establish  at  least  two  competitive  sources  for  production.  This 
law  was  later  amended  in  November  of  1990.  [Ref.  23] 
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The  purpose  of  a  teaming  arrangement  is  to  avoid  reliance  on  a  sole  source  of 
production  for  the  system.  Each  contractor  is  to  provide  his  particular  area  of  expertise 
for  the  program  and  have  the  capability  to  independently  produce  the  system  after 
development.  In  the  A- 12  Program,  the  contractors  established  a  written  agreement  that 
provided  each  team  member  would  try  to  convince  the  Navy  to  guarantee  a  minimum  of 
40  percent  of  the  production  to  each  team  member.  When  the  requirement  for  the  A- 12s 
was  reduced  to  620  aircraft  a  competitive  production  contract  was  not  feasible. 

According  to  a  General  Accounting  Office  (GAO)  report  issued  in  1992,  "...an 
award  split  other  than  50-50  or  60-40  would  have  put  the  losing  contractor  out  of  the 
A-12  business."  [Ref.  23:  p.  7], 

The  larger  member  of  the  A-12  team,  McDonnell  Douglas,  was  created  in  1967 
with  the  acquisition  of  the  financially  troubled  Douglas  Aircraft  Company  by  McDonnell 
Company.  McDonnell  Douglas  has  been  the  Nation's  top  prime  contractor  in  all  but  one 
of  the  years  from  1984  to  1994  [Ref.  8],  Its  average  share  of  the  Department  of  Defense's 
prime  contracts  has  averaged  around  6  percent  of  the  value  of  total  contracts  awarded. 
According  to  Financial  World  magazine  McDonnell  Douglas  has  a  market  value  of  $5.6 
billion,  154th  in  value  of  all  US.  companies.  In  1994  McDonnell  Douglas  had  $13.3 
billion  in  sales  and  $562  million  in  profits  [Ref.  13], 

McDonnell  Douglas  produces  the  F- 15  Eagle,  F-18  Hornet,  C-17  Globemaster  III, 
AV-8  Harrier,  T-45  Goshawk,  AH-64  Apache,  and  the  Tomahawk  and  Harpoon  missiles. 
It  also  has  RDT&E  contracts  for  electronics  and  communications  equipment,  along  with 
missile  and  space  systems.  [Ref.  5] 

The  other  member  of  the  A-12  team,  General  Dynamics,  has  consistently  been  in 
the  top  ten  Defense  contractors  for  the  past  ten  years.  Its  dollar  value  of  prime  contracts 
has  ranged  from  $2  billion  to  $8  billion.  Its  market  value  is  $2.75  billion,  placing  it  at 
number  329  of  the  nation's  top  companies.  In  1994  it  had  $3  billion  in  sales  and  $214 
million  in  profits.  [Ref.  13] 

General  Dynamics  produces  the  F-16  Falcon,  the  Seawolf  nuclear  submarine,  M-l 
tank,  Tomahawk  missile.  Stinger  missile,  RIM-66  missile  and  the  MK-15  Close  in  Weapon 
System.  [Ref.  5] 
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Both  companies  have  a  history  of  litigation  actions  with  the  Government.  They 
currently  are  contesting  the  $1.35  billion  in  unliquidated  progress  payments  demanded  by 
the  Government  and  they  have  a  certified  claim  for  equitable  adjustment  for  $1.4  billion 
resulting  from  the  termination  of  the  A- 12  Program  [Ref.  28,  July  27  1994],  A  ruling  in 
December  1994  that  vacated  the  Department  of  Defense's  Termination  for  Default  was 
considered  a  victory  by  the  contractors  in  their  efforts  to  clear  their  financial  debt  to  the 
Government  on  the  A- 12  Program. 

E.  TERMINATION 

1.  Overview 

The  A- 12  Program  was  terminated  for  default  on  7  January  1991.  The  Secretary 
of  Defense  directed  the  Navy  to  terminate  the  troubled  Program.  Although  many  reasons 
contributed  to  the  termination  of  the  Program,  this  research  is  primarily  concerned  with 
the  stated  reason  for  termination:  cost  overruns.  The  estimate  at  completion  for  the  A- 12 
varied  significantly.  The  Secretary  of  Defense  stated  uncertainty  in  estimating  the  cost  of 

the  contract  was  the  primary  reason  for  terminating  the  Program. 

This  program  cannot  be  sustained  unless  I  ask  Congress  for  more 
money  and  bail  the  contractors  out,  but  I  have  made  the  decision  that  I  will 
not  do  that.  No  one  can  tell  me  exactly  how  much  more  money  it  will  cost 
to  keep  this  program  going.  I  do  not  believe  a  bailout  is  in  the  national 
interest.  If  we  cannot  spend  the  taxpayers'  money  wisely,  we  will  not 
spend  it.  [Ref.  22:  p.  3] 

2.  Costs  at  Termination 

The  cost  of  the  work  accomplished  in  the  A- 12  Program  is  difficult  to  estimate 
accurately.  Under  a  fixed-price  type  contract  the  contractor  is  required  to  provide  a 
deliverable  to  the  Government.  The  Government  will  pay  for  the  products  or  services  it 
has  accepted. 

In  the  A- 12  Program,  the  value  of  the  work  accomplished  was  priced  at  $1 .34 
billion.  The  contractors  had  incurred  much  greater  costs  in  the  development  of  the 
aircraft.  Table  4  provides  a  detailed  accounting  of  work  accepted  by  the  Government  at 
termination. 
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ITEM 

Price 

(In  Millions) 

Initial  Design  Review 

269.8 

Preliminary  Design  Review 

261.9 

Critical  Design  Review  (Engine) 

139.0 

Critical  Design  Review  (Phase  IA) 

150.6 

Phase  1 A  Test  Review 

197.0 

Program  Management  Review 

316.3 

Total 

$1,334.7 

Table  4.  Value  of  Work  Accepted.  [Ref.  26] 


While  the  value  of  work  accepted  was  priced  at  $1 .33  billion,  the  actual  price  paid 
for  the  work  was  $2.69  billion.  Of  this  $2.69  billion  the  Government  has  demanded 
$1,352  billion  in  unliquidated  progress  payments  be  returned.  The  contractors  are 

currently  disputing  the  decision  of  the  Government  of  the  value  of  work  accepted.  [Ref. 
25], 


Total  expenditures  on  the  A- 12  Program  exceeded  the  value  of  the  work  accepted 
at  termination.  Table  5  provides  a  listing  of  expenditures  on  the  A- 12  Program  at  the 
point  of  termination. 


CATEGORY 

$ 

(Billions) 

Total  Appropriations 

6.7 

Total  RDT&E  Expenditures 

2.85 

FSD  Contract  Expenditures 

2.56 

Long  Lead  Procurement 

0.104 

Value  of  Work 

1.34 

Unliquidated  Progress  Payments 

1.35 

Table  5.  A- 12  Program  Expenditures.  [Ref.  26] 


3.  Contractor  Claims 

The  contractors  have  an  immense  financial  interest  in  the  favorable  settlement  of 
the  litigation.  The  conversion  of  the  termination,  from  default  to  convenience,  would 
completely  restructure  the  settlement  amounts. 

The  contractor  team  has  filed  a  claim  for  equitable  adjustment  of  $  1 .4  billion.  The 
contractors  claimed  that  the  contract  could  not  be  completed  according  to  the 
contractually  specified  terms  because  of: 
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1 .  The  Navy's  superior  knowledge  of  facts  vital  to  performance. 

2.  Delays  and  disruptions  caused  by  the  Navy's  conduct. 

3.  The  Navy's  flawed  acquisition  strategy. 

4.  Commercial  impossibility  of  performance. 

The  $1 .35  billion  demanded  by  the  Government  has  been  deferred  pending  the 
outcome  of  the  litigation.  [Ref.  25] 

F.  THE  JOINT  ADVANCED  STRIKE  TECHNOLOGY  PROGRAM 

1.  Overview 

The  Navy  committed  seven  years  and  $2,985  billion  to  the  A-12  Program.  If  the 
contractors  are  successful  in  their  litigation  the  cost  of  the  A-12  Program  to  the 
Government  could  go  as  high  as  $6  billion.  At  the  termination  of  the  Program  the  Navy 
had  very  little  that  it  could  use  in  a  future  development  program.  During  hearings  by  the 
Committee  on  Armed  Services  of  the  House  of  Representatives  on  the  A-12  termination. 
Congressman  Sisisky  called  the  termination  of  the  A-12  Program  "..a  national  disaster 
with  naval  aviation." 

2.  The  Next  Fighter 

The  cost  of  developing  a  fighter  aircraft  to  fulfill  just  the  Navy's  needs  is 
prohibitively  expensive.  A  joint  program  with  the  Air  Force  is  currently  ongoing  called 
the  Joint  Advanced  Strike  Technology  Program  (JAST). 

The  JAST  aircraft  is  being  designed  as  a  low  observable  technology  aircraft  that  is 
projected  to  replace  the  Navy's  A-6,  the  USAF's  F-16  and  the  Marines  AV-8.  While  the 
JAST  aircraft  will  not  have  the  ordinance  carrying  capacity  that  was  provided  by  the  A-6, 
this  shortcoming  is  expected  to  be  made  up  for  in  its  use  of  smart  weapons  and  increased 
survivability. 

The  JAST  Program  started  in  1994  and  is  currently  in  Phase  0  of  the  acquisition 
process.  It  is  expected  to  start  EMD  in  the  year  2000,  Low  Rate  Initial  Production 
(LRIP)  in  2005,  and  Initial  Operational  Capability  by  the  year  2007.  [Ref.  16] 

The  total  cost  (life  cycle  cost)  of  the  Program  is  expected  to  range  from  $160  to 
$200  billion.  Unit  procurement  costs  are  expected  to  be  from  $30  to  $40  million.  This 
figure  is  relatively  inexpensive  when  compared  to  the  cost  of  other  currently  operational 
combat  aircraft  (See  Figure  5). 
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The  JAST  aircraft  is  being  conceived  of  as  a  modular  aircraft  that  will  utilize  a 
common  fuselage  and  Service  unique  components  would  then  be  added  on.  According  to 
the  Program  Office,  85  percent  commonality  of  parts  is  the  goal.  [Ref.  16] 

The  Mission  Need  Statement  and  the  Operational  Requirements  Document  from 
the  A- 12  Program  were  used  as  the  base  for  developing  the  MNS  and  ORD  for  the  JAST 
Program.  The  Program  Manager  is  an  0-7.  The  duties  of  Program  Manager  will  rotate 
between  the  USAF  and  the  Navy.  [Ref.  16] 

G.  SUMMARY 

Chapter  III  provided  a  short  summary  of  the  A- 12  Program.  The  Program 
generally  followed  the  basic  Acquisition  Process  outlined  in  Chapter  II.  Although  the 
A- 12  was  a  restricted  access  program  it  was  still  subject  to  oversight.  Shortly  after  the 
start  of  the  FSD  contract  the  Program  started  incurring  cost  overruns.  As  the  Program 
continued  the  overruns  became  worse  The  uncertainty  in  the  estimates  of  final 
completion  costs  resulted  in  the  termination  of  the  Program. 

Chapter  IV  presents  comparison  data  of  other  EMD  programs  and  cost  variances. 
These  data  are  used  to  contrast  the  A- 12  cost  variances  and  determine  if  the  A-12's 
overruns  were  different  than  other  major  acquisitions. 
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IV.  DATA 


A.  OVERVIEW 

This  chapter  provides  the  data  that  are  used  to  answer  the  primary  research 
question:  Was  the  A- 12  Program  over  cost  when  compared  to  other  similar  acquisitions? 
Chapter  IV  begins  by  providing  the  actual  reported  cost  data  for  the  A- 12  Program  up  to 
the  point  of  contract  termination.  Data  from  58  comparison  contracts  are  then  provided 
as  the  sample  to  be  tested  against. 

There  are  hundreds  of  contracts  in  the  DAES  database.  The  selection  of  the 
contracts  used  in  this  research  conformed  to  the  requirements  set  out  in  Chapter  I,  with  a 
few  exceptions.  Sixty-five  EMD  contracts  were  initially  drawn  from  the  database.  Seven 
contracts  were  eliminated  from  the  sample  due  to  excessive  rebaselining  or  insufficient 
data  points.  Three  contracts  were  retained  that  did  not  meet  the  ten  data  point 
requirement.  These  contracts  were  for  aircraft  development  and  met  the  other  specified 
requirements  for  inclusion. 

It  became  apparent,  as  this  research  was  conducted,  that  all  programs  experience 
adjustments  to  the  Contract  Budget  Base  (CBB).  In  the  58  contracts  examined  for 
comparison  with  the  A- 12  Program  only  two  contracts  showed  a  downward  adjustment  in 
the  CBB.  In  one  case  the  program's  funding  was  reduced  by  nearly  half,  the  other 
contract  showed  a  cost  at  completion  that  was  just  slightly  under  the  original  estimate. 

The  effects  of  inflation  do  not  account  for  the  upward  revisions.  Contractor 
expenditure  budgets  are  based  on  future  predictions  of  price  escalation.  This  was 
confirmed  by  a  former  program  manager  of  Lockheed  Corporation  and  a  Cost  Analyst 
with  the  CAIG.  The  explanations  for  the  adjustments  to  the  CBB  can  include  poor 
original  understanding  of  the  scope  of  work,  or  adjustments  to  the  contractual 
requirements.  Regardless  of  the  cause,  only  five  of  the  58  contracts  examined  finished  the 
EMD  phase  within  five  percent  of  the  original  CBB. 
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B.  A-12  PROGRAM  COST  DATA 
1.  Program  Information 

The  A-12  Program  began  the  EMD  phase  in  January  1988.  Significant  effort  on 
the  Program  was  expected  to  be  complete  by  April  1996,  with  first  flight  of  a  prototype 
aircraft  in  June  1990.  During  the  life  of  the  A-12  Program,  27  Cost  Performance  Reports 
and  nine  Selected  Acquisition  Reports  (SAR)  were  submitted.  The  data  from  the  A-12 
Program  will  be  compared  to  the  sample  58  contracts  to  determine  if  the  A-12  cost 
variances  were  within  the  population  of  cost  variances. 


The  comparison  between  the  A-12  cost  variances  and  the  sample  is  based  on  cost 
variance  at  several  percentage  completion  points.  Table  6  provides  the  reported  A-12  cost 
information. 


REPORT 

DATE 

BCWP 

ACWP 

% 

CV 

% 

COMPLETE 

CBB 

Sept  1988 

“7.39 

7,6 

3981! 

Dec  1988 

441.7 

494.5 

-11.95 

11.1 

3981! 

March  1989 

689.4 

—1 5.94' : 

14.9 

3981.1 

Sept  1989 

1009.6 

1220.5 

-20.89 

25.4 

3982.2 

Dec  1989 

mm 

-24,85 

31,6 

4042.3 

Apr  1990 

1491 

1950 

-30.78 

36.8 

4046.4 

June  1990 

2285,5 

-34,09 

4047! 

Sept  1990 

1881.5 

2611 

-38.77 

46.5 

4044.7 

Nov  1990  1990.6  2789 

Note:  Figures  are  reported  in  millions  of  constant  dollars. 

-40.11 

49.2 

4045 

Table  6.  A-12  Reported  Cost  Information. 


The  A-12  cost  data  are  drawn  from  the  nine  submitted  Selected  Acquisition 
Reports.  The  percent  complete  and  the  percent  cost  variance  are  as  reported.  The 
Program  was  terminated  in  January  1991.  The  accuracy  of  the  reported  cost  data  is  of 
great  importance  in  determining  whether  the  A-12  Program  differed  significantly  from 
other  similar  programs. 
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2.  Accuracy  of  Reported  Cost  Data 

The  accuracy  of  the  A- 12  ACWP  data  can  be  affected  by  numerous  factors.  There 
is  delay  from  the  time  work  has  occurred  to  when  it  is  reported.  There  can  also  be 
material  understatements  of  the  actual  cost  of  the  work  performed. 

During  the  EMD  phase  of  the  A- 12  Program,  McDonnell  Douglas  was 
experiencing  cash  flow  problems  [Ref.  15],  Significant  cost  variances  reduce  the  amount 
of  the  progress  payments  based  on  estimates  at  completion.  A  reduction  in  the  amount  of 
the  progress  payments  made  to  McDonnell  Douglas  would  have  had  a  negative  financial 
impact  in  what  was  already  a  poor  financial  climate  for  the  contractor.  Incentives  exist  to 
under  report  cost  information  on  an  over-budget  program,  or  to  shift  costs  from  an 
over-budget  program  to  a  within-budget  program. 

Evidence  of  McDonnell  Douglas  shifting  costs  from  the  A- 12  Program  to  other 
Government  contracts  was  reported  in  the  September  1 1,  1995  edition  of  the  Wall  Street 
Journal.  According  to  the  article,  the  U.S.  Justice  Department  has  decided  to  litigate  a 
whistle  blower  case.  It  is  alleged,  "...company  supervisors  systematically  ordered 
assembly-line  employees  to  improperly  shift  untold  millions  of  dollars  of  labor  costs 
between  various  weapons  programs,  including  the  C-17  cargo  plane,  F-18  fighters  and  the 
Navy's  now-cancelled  A- 12  carrier  based  stealthy  attack  jet."  The  article  further  states 
that  the  C-17  and  the  A- 12  were  "the  biggest  beneficiaries  of  the  alleged  cost  shifting." 
The  amount  of  money  involved  is  in  the  hundreds  of  millions  of  dollars.  [Ref.  28,  1 1  Sept. 
1995] 

No  correction  can  empirically  be  made  to  the  actual  reported  costs  for  the 
Program.  If  the  ACWP  figures  have  been  understated  it  can  not  be  shown  within  the 
bounds  of  this  research,  and  the  impact  it  may  have  had  on  the  Program  is  also  out  of  the 
scope.  An  assumption  is  made  that  cost  shifting  may  occur  in  any  given  program  and  no 
adjustment  is  made  to  the  data. 

3.  The  Problem  of  Rebaselining 

The  percentages  complete  of  the  A- 12  Program  as  shown  in  Table  6  is  based  on  a 
Contract  Budget  Base  of  $3.98  to  $4.05  billion.  Since  the  contract  would  have  exceeded 
the  ceiling  price  of  $4.7  billion  the  Program  would  have  been  rebaselined  if  it  had 
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continued  to  completion.  No  significant  adjustments  were  made  to  the  CBB  in  the  A- 12 
Program.  In  all  the  comparison  contracts  some  degree  of  rebaselining  occurred,  in  many 
cases  the  amount  of  the  rebaseline  was  quite  significant.  Table  7  shows  the  average 
percentage  adjustments  in  the  sample  program's  CBBs.  For  contract  information,  the 
reader  is  referred  to  Appendix  A.  To  determine  the  CBB  adjustments  a  relatively  large 
percentage  of  the  program  had  to  be  complete.  The  percentage  was  determined  to  be  data 
points  covering  at  least  50  percent  of  the  total  program,  and  a  program  completion  point 
achieved  of  at  least  70  percent.  The  adjustment  is  determined  by  measuring  the  difference 
between  the  initial  CBB  and  the  ending  CBB. 


MEAN 

MEDIAN 

MAX 

MIN 

N 

Fixed  Price 

12G% 

Cost  Type 

57% 

45% 

232% 

-4% 

22 

Aircraft 

•••  • 

•  80%' 

232% 

445% 

Total  Sample 

54% 

46% 

232% 

-61% 

29 

Table  7.  Contract  Budget  Base  Adjustments. 


Adjustments  to  the  CBB  have  two  major  affects  on  the  reported  cost  information 
First,  the  percentage  completion  is  reduced  based  on  the  new  budget.  Second,  the  cost 
variance  percentage  changes,  based  on  the  new  BCWP  figures. 

Both  affects  impact  the  ability  to  estimate  final  completion  costs  accurately. 
Adjustments  to  the  contract's  budget  can  be  made  for  a  variety  of  reasons.  The  more 
stable  the  baseline,  the  more  accurate  the  estimate  at  completion  will  be. 


Table  8  provides  adjusted  percentage  completion  points  of  the  A- 12  Program 
based  on  three  time  estimates  to  complete. 


ESTIMATED 

1ST 

FLIGHT 

ESTIMATED 

COMPLETION 

DATE 

K 

LENGTH 

(Mo*s) 

AT  TERM 
%  COMP 

EAC 

dune  1990 

100 

4777 

June  1992 

Sept  1999 

142 

24 

9069 

Dec  1992 

Jan  2001 

158 

11061 

Note:  Figures  are  reported  in  millions  of  constant  dollars. 

Table  8.  Rayleigh  Model  Estimates  of  A- 12  Program  Expenditures. 
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The  first  time  estimate  is  the  contractually  specified  first  flight  and  contract 
length.  The  second  and  third  estimates  are  based  on  estimates  of  the  first  flight  date,  at 
contract  termination.  The  first  flight  dates  were  obtained  from  Reference  9.  The  first 
flight  date,  in  months  from  program  start,  is  multiplied  by  2.65  [Ref.  18]  to  estimate  the 
total  program  length.  The  estimates  at  completion  were  derived  using  the  Rayleigh  model 
methodology  described  in  Chapter  II.  The  percentage  complete  at  termination  is 
determined  by  taking  the  termination  cost  of  the  A- 12  and  dividing  it  by  the  EAC.  The 
EAC  is  based  on  an  assumption  of  40  percent  of  costs  incurred  at  first  flight. 

4.  The  Estimate  at  Completion 

The  accuracy  of  using  the  Rayleigh  model  to  estimate  cost  at  completion  was 
tested  by  Elrod  [Ref.  11].  Elrod  found  a  generally  good  fit  between  the  Rayleigh  model 
and  actual  costs  at  completion.  Gallagher,  a  cost  analyst  in  the  Cost  Analysis 
Improvement  Group  (CAIG),  further  developed  this  technique  [Ref.  14]. 

The  Rayleigh  model's  predictive  capability  was  tested  against  ten  aircraft  program 
contracts.  A  comparison  was  then  made  using  the  CPIcum  technique  to  determine  the 
more  accurate  predictor.  The  results  of  this  test  are  provided  in  a  graph  in  Figure  6. 
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As  shown  in  Figure  6,  the  Rayleigh  estimates  provide  significantly  less  variance 
from  actual  completion  costs  over  the  life  of  the  program.  To  derive  the  curves  in  the 
graph,  data  points  from  all  ten  programs  were  plotted  against  time  and  an  exponential 
curve  was  fit  to  the  data.  Generally,  past  the  20  percent  completion  point,  the  Rayleigh 
model  provides  reasonable  estimates  of  actual  program  completion  costs.  The  percentage 
cost  variances  are  within  10  percent  of  actual  ending  costs  at  the  40  percent  completion 
point.  Using  the  CPIcum  technique,  the  estimates  are  not  within  10  percent  of  actual 
ending  costs  until  the  80  percent  completion  point.  On  the  basis  of  Elrod  and  Gallagher's 
work,  and  the  test  against  ten  actual  contracts,  the  Rayleigh  model  is  accepted  as 
providing  a  realistic  estimate  of  A- 12  Program  costs. 

5.  Likelihood  of  the  Estimates 

An  estimate  of  a  future  occurrence  is  by  definition  uncertain.  It  is  of  interest  to  be 
able  to  say  with  what  probability  the  outcome  will  occur.  The  estimates  of  the  final  cost 
of  the  A- 12  Program  in  Table  8  were  tested  using  a  software  program  developed  and 
provided  by  Gallagher  [Ref.  14]  to  determine  the  likelihood  that  the  ending  cost  estimate 
would  occur.  Gallagher's  work  uses  past  cost  figures  to  develop  a  Bayesian  probability  of 
future  occurrence  of  the  ending  cost.  He  developed  this  technique  while  working  as  a  cost 
estimator  with  the  C  AIG. 

Three  estimates  at  completion  were  tested  using  Gallagher's  model.  The  first 
estimate  was  the  Program  Manager's  EAC  at  contract  termination.  The  second  and  third 
estimates  were  the  EACs  produced  by  the  Rayleigh  model  for  first  flight  dates  of  June 
1992  and  December  1992.  Table  9  gives  the  probability  that  final  ending  costs  would 
exceed  the  estimate.  For  a  detailed  explanation  of  the  development  of  the  probabilities 
and  the  Rayleigh  estimates,  the  reader  is  referred  to  Final-Cost  Estimates  for  Research  & 
Development  Programs  Conditioned  on  Realized  Costs  by  Mark  A.  Gallagher  and  David 
A.  Lee  [Ref.  14],  The  article  is  available  through  the  Defense  Technical  Information 
Center  database. 
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1ST  FLIGHT 

PROBABILITIES  OF  EACs 

P(X>=4.7B) 

P(X>=7.5B) 

P(X>9.07B) 

P(X>=11B 

June  1992 

0.99 

0:84 

0.63 

0:37 

Dec  1992 

0.99 

0.87 

0.70 

::  0.49 

Table  9.  Probability  of  Final  Cost  Estimates. 


In  Table  9,  the  cumulative  probabilities  of  occurrence  of  the  given  cost  estimate 
are  provided.  The  first  column,  1st  Flight,  presents  two  possible  first  flight  dates.  These 
dates  are  drawn  from  the  CAIG  estimates  provided  to  the  DoD  IG.  Columns  two  through 
four  provide  four  possible  EACs  and  the  probabilities  of  the  Program  exceeding  the  given 
EAC.  The  first  EAC  is  the  original  ceiling  price  of  the  contract.  The  probability  that  the 
Program  would  have  exceeded  this  ceiling  is  almost  certain.  If  first  flight  would  have 
occurred  in  December  1992,  the  probability  that  the  Program  cost  would  have  exceeded 
$11  billion  is  .49. 

To  compute  the  probabilities  shown  in  Table  9,  two  time  assumptions  are  required. 
For  a  first  flight  date  of  June  1992,  the  minimum  Program  length  was  assumed  to  be  five 
years  from  initiation  with  the  maximum  length  of  13  years.  If  the  first  flight  would  have 
occurred  in  December  of  1992,  then  the  minimum  length  was  assumed  to  be  eight  years 
with  a  maximum  length  of  15  years. 

C.  COMPARISON  CONTRACT  COST  DATA 

1.  The  Comparison  Sample 

Over  1 00  contracts  were  examined  from  the  DAES  database  for  inclusion  into  the 
sample.  Sixty-five  contracts  were  initially  found  to  meet  the  requirements  as  specified  in 
Chapter  I:  at  least  ten  data  points  existed,  the  program  had  achieved  a  significant  level  of 
completion,  and  the  program  was  classified  as  an  AC  AT  ID  program.  Of  the  65  contracts 
selected  for  comparison,  five  were  eliminated  due  to  non-consistency  in  completion 
percentages  and  two  contracts  were  eliminated  due  to  insufficient  data. 

2.  Characteristics  of  the  Sample 

Fifty-eight  contracts  constitute  the  sample.  To  be  included  in  the  DAES  database 
the  program  must  be  an  ACAT  ID  program.  All  the  contracts  examined  were  for  major 
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programs.  The  completion  points  of  the  sample  ranged  from  a  low  of  40  percent  complete 
to  a  high  of  100  percent  complete.  The  average  completion  point  achieved  by  the  58 
sample  contracts  was  83  percent  complete.  Table  10  shows  the  characteristics  of  the 
sample  examined. 


FIXED 

PRICE 

COST 

REIMB 

TOTAL 

ARMY 

X,-  ■  I-:-.:  ".S 

18 

19  - 

AF 

13 

7 

20 

NAVY 

'■  •  2 

TOTAL 

16 

32 

58 

Table  10.  Characteristics  of  the  Sample. 


3.  Sample  Data 

The  cost  variances  of  the  sample  were  determined  using  equation  (2.3).  All 
contracts  that  met  the  specified  criteria  in  Chapter  I  were  used.  Not  all  the  sample 
contracts  had  data  points  for  each  of  the  percentage  completion  points.  For  those 
contracts  that  had  multiple  data  points  within  a  percentage  completion  range,  the  cost 
variances  were  averaged  to  avoid  weighting  data  from  any  one  contract  more  than  other 
contracts. 

The  data  for  the  percentage  completion  brackets  are  each  modeled  seperately. 

Only  the  0  to  10  percentage  completion  bracket  exhibits  the  Normal  distribution  that  one 
would  expect  if  cost  variances  were  distributed  about  a  mean  cost  variance  of  zero. 

Table  1 1  provides  the  cost  variances  observed  in  the  sample  contracts.  In  Table 
1 1,  the  completion  point  refers  to  the  percentage  complete  the  program  has  achieved 
measured  by  time  expended  divided  by  total  time.  Column  2  gives  the  arithmetic  mean  of 
the  sample.  The  data  are  skewed  to  the  left.  The  median  is  provided  as  a  reference  point. 
The  sample  of  all  58  contracts  is  modeled  by  the  Beta  distribution.  Column  4  indicates  the 
distribution  that  modeled  the  data  using  a  Chi-squared  test. 
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COMPLETION 

POINT 

MEAN 

MEDIAN 

TYPE 

DISTRIBUTION 

N 

040% 

•  +1.78% 

+  1;68% 

;  :r:- ::  ^ 

WrnrnWmMl 

11-20% 

-6.35% 

-2.68% 

BETA 

37 

-^a09&.v 

V  : -5,27%  ••  ,v: 

-2.75% 

31-40% 

-8.18% 

-4.88% 

LOGISTIC 

37 

;  41-50% 

■41,16% 

-6:95% 

\  '•  43 

51-60% 

-10.23% 

-4.23% 

LOGISTIC 

43 

61-70% 

40.59% 

-3,85% 

42 

71-80% 

-14.52 

-6.68% 

BETA 

45 

43,08% 

-7,02% 

BETA 

>90% 

-11.16% 

-4.93% 

BETA 

27 

4 3,37% 

-6.09% 

BETA 

WmMMmm 

Table  1 1 .  Cost  Variances  of  Sample  Contracts. 


4.  Aircraft  Program  Data 

Seventeen  contracts  were  included  in  the  aircraft  cost  variance  sample.  Three 
contracts  for  aircraft  developmental  work  were  included  in  the  sample  that  had  only  nine 
data  points.  To  be  included  in  the  sample  the  program  had  to  show  relatively  increasing 
degrees  of  completion  as  work  progressed.  The  aircraft  program  contracts  consisted  of 
the  contract  for  the  program  itself  or  for  major  aircraft  type  subassemblies.  Contracts  for 
subassemblies  such  as  avionics,  engines  or  armaments  were  not  considered.  Contracts  for 
follow-on  developmental  work  were  also  not  included.  Helicopters  were  considered  as 
aircraft. 

Of  the  19  contracts  initially  examined  only  one  program  was  terminated.  The 
remaining  programs  continued  into  production  or  are  still  in  the  EMD  phase.  The  one 
program  that  was  terminated  was  later  restarted  under  a  different  designation.  This 
contract  was  not  considered  for  inclusion  since  many  of  the  technical  problems  that  caused 
the  original  termination  would  have  been  paid  for  and  would  not  have  shown  up  in  the 
new  ACWP  figures.  Table  12  provides  the  mean  and  median  cost  variances  of  EMD 
aircraft  programs. 
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COMPLETION 

POINT 

MEAN 

CV 

MEDIAN 

CV 

N 

0-10% 

-2.99% 

•  -4% 

•  11 

11-20% 

-8.12% 

-5.18% 

11 

21-30% 

-4.05% 

31-40% 

-8.38% 

-5.34% 

14 

41-50% 

-13.58% 

51-60% 

-12.51% 

-8.99% 

12 

61-70% 

-16.88% 

-8.15% 

iiiliil:; 

71-80% 

-16.79% 

-11.33% 

13 

81-90%  : 

-11.96% 

-6.09% 

>90% 

-17.93% 

-9.01% 

8 

ALL 

-18.4% 

-6.13% 

Table  12.  Aircraft  Program  Cost  Variances. 


The  mean  cost  variance  for  all  aircraft  programs  examined  was  -18.4  percent.  The 
median  value  was  -6.13  percent.  Seventeen  programs  were  included  in  the  sample. 

5.  Fixed-Price  Versus  Cost-Type  Contracts 

All  contracts  that  had  exceeded  50  percent  complete  were  included  in  the  sample. 
The  mean  cost  variance  for  all  programs  examined  was  -16.5  percent.  Table  13  provides 

comparison  data  for  mean  overruns  of  fixed-price  type  contracts  versus  cost-type 
contracts. 


TYPE 

CONTRACT 

MEAN 

CV% 

MEDIAN 

CV% 

N 

ilpIgiKt 

-21.02% 

COST 

-14.76% 

-6.05% 

39 

TOTAL 

-16.5% 

,  ■  -6.78% : 

54 

Note:  Totai=Most  recent  cv  for  contracts 

>  50%  complete. 

Table  13.  Fixed-Price  Versus  Cost-Type  Contract  Cost  Variances. 
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D.  SUMMARY 

This  chapter  presented  the  data  to  be  used  in  Chapter  V  to  determine  if  the  A- 12 
Program  was  significantly  different  from  other  EMD  programs.  The  difficulty  of 
developing  accurate  estimates  at  completion  is  caused  by  several  factors  such  as 
rebaselining,  changes,  or  extensions  of  program  length.  The  mean  cost  variance  of  EMD 
programs  is  -16.5  percent  with  a  median  value  of  -6.78  percent.  While  the  median  value 
for  cost  variances  does  not  appear  to  be  too  significant,  the  true  cost  variances  are  masked 
by  adjustments  to  the  CBB.  The  mean  adjustment  to  the  CBB  is  54  percent  with  a  median 
value  of  46  percent. 

Adjustments  to  the  CBB  and  cost  variances  are  evaluated  in  Chapter  V.  Chapter 
V  tests  the  hypothesis  that  the  A- 12  Program  cost  variances  were  no  greater  than  other 
EMD  programs.  The  next  chapter  also  examines  secondary  causes  for  the  termination  of 
the  A- 12  Program. 
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V.  DATA  ANALYSIS 


A.  OVERVIEW 

Chapter  IV  presented  the  data  from  58  separate  EMD  contracts.  Chapter  V 
establishes  percentile  values  for  the  sample  programs  and  compares  the  A- 12  Program 
cost  variances.  Initially,  the  A- 12  is  compared  to  the  entire  sample  for  differences.  The 
A- 12  cost  variances  are  then  compared  to  other  aircraft  programs. 

B.  A-12  COST  VARIANCES 

I.  Percentile  Values 

To  provide  a  valid  comparison  of  the  sample  data  and  the  A-12  cost  variances,  the 
cost  variances  of  the  sample  are  grouped  by  percentage  completion  points.  Percentile 
values  are  determined  for  each  sample  point  based  on  the  type  of  distribution.  The 
procedure  to  accomplish  this  was  to  use  the  statistical  software  program.  Minitab  [Ref. 
19].  The  Inverse  CDF  command  returns  the  percentile  value  for  the  specified  point.  With 
the  Beta  distribution  the  critical  value  is  a  value  from  0  to  1 .  This  value  is  converted  to 
the  actual  value  using  techniques  described  in  Appendix  B.  Table  14  provides  the 
percentile  values  for  the  designated  percentage  completion  points. 


% 

COMPLETE 

TYPE 

DISTRO 

PARAM 

1 

PARAM 

2 

CHI 

SQD 

VALUE 

95th 

PERCENTILE 

VALUE 

99th 

PERCENTILE 

VALUE 

0-10 

Normal 

::  L79 

11.56 

17,56 

-17.22 

11  -20. 

Beta 

2.3 

1.07 

16.54 

-23.85 

-29.33 

21-30 

Logistic 

-5,27 

6:08 

26.84: 

-23.17  ,  ... 

31  -40 

Logistic 

-8.08 

5.54 

26.04 

-24.39 

-33.53 

41-50 

Logistic 

8.08 

49^92 

>34,81; 

51  -60 

Logistic 

-9.99 

7.81 

37.77 

-32.98 

-45.88 

61  -  70 

Beta 

.  •  1-67: 

25^25 

-75.2 

71  -80 

Beta 

2.53 

1.18 

38.33 

-46.29 

-56.23 

81  -90 

.  Beta 

1.93 

0.78 

-49.66 

?i|i60,1:9. 

>90 

Beta 

1.06 

0.22 

10 

-37.52 

-47.73 

AII>8G% 

Beta 

1.49 

0.82 

•y21'.34-x-:>;' 

-46.98 

-53.65 

Table  14.  Percentile  Values  for  the  Sample  Contracts. 
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The  data  from  Table  14  are  used  to  answer  the  primary  research  question  and 
Hypothesis  1 :  Point  values  of  the  A- 12  are  no  different  from  cost  variances  of  the  sample. 
The  first  column  of  the  Table  provides  the  percentage  completion  points  of  the  sample 
contracts.  Column  2  provides  the  distribution  that  best  modeled  the  data,  the  resulting 
Chi-squared  value  is  in  Column  5.  The  parameters  associated  with  the  distribution  are  in 
Columns  3  and  4.  For  the  Normal  distribution,  the  parameters  are  given  by  the  mean  and 
standard  deviation.  The  percentile  values  are  the  area  of  the  distribution  above  the  given 
value. 

2.  A-12  Cost  Variance  Percentiles. 

After  determining  the  critical  values  in  Table  14,  the  next  step  was  to  evaluate 
where  the  A-12  cost  variances  were  in  relation  to  the  distribution.  Table  15  provides  the 
comparison  of  A-12  cost  variances  versus  the  sample. 


% 

COMPLETE 

% 

CV 

TYPE 

DISTRO 

AREA 

ABOVE 

VALUE 

-7.39 

Normal 

0.7907 

11.1 

-11.95 

Beta 

0.7305 

14.9 

-15.94 

Beta 

0,8257 

25.4 

-20.89 

Logistic 

0.9288 

319 

i  -24.85 

Logistic 

0.847 

36.8 

-30.78 

Logistic 

0.9837 

42.1 

-34.09 

Logistic 

09456 

46.5 

-38.77 

Logistic 

0.9688 

...  a-'4&2  V.  •• 

-40.11 

Logistic 

019734 

vs.samp 

-40.11 

Beta 

0.9108 

Table  15.  A-12  Cost  Variance  Percentile  Values. 

The  results  from  Table  15  agree  with  prior  work  by  Heise  [Ref.  18],  Christensen 
[Ref.  3],  and  observations  made  by  Abba  and  Christie.  As  the  program  proceeds  the  cost 
overrun  does  not  improve. 
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In  Table  1 5,  Columns  1  and  2  are  the  percentage  completion  points  and  cost 
variance  as  reported.  The  distribution  that  modeled  the  data  is  in  Column  3.  The  final 
Column  provides  the  percentage  of  programs  that  the  A- 12  cost  variance  exceeded. 

The  A- 12  cost  variances  exceeded  75  percent  of  sample  programs  at  the  7.6 
percent  completion  point.  The  cost  variances  worsened  from  that  point  on.  Given  Heise's 
earlier  findings  that  the  CPIcum  will  remain  stable  from  the  20  percent  completion  point 
on,  it  is  unlikely  that  the  cost  variances  of  the  A- 12  would  have  improved,  if  the  program 
had  continued.  The  relation  of  the  A-12's  cost  variances  to  other  programs  did  not 
improve.  At  termination  the  cost  variance  of  the  A- 12  exceeded  97  percent  of  all  other 
programs.  As  shown  from  the  sample  data,  the  A- 12  cost  overruns  were  exceptional. 

C.  AIRCRAFT  PROGRAMS 

1.  Aircraft  Program  Percentile  Values 

In  Table  16,  aircraft  program  cost  variances  are  examined  to  determine  if  A- 12 
cost  variances  differed  from  other  aircraft  programs.  The  same  analysis  techniques  were 
used  as  with  the  comparison  to  the  overall  sample. 


% 

COMPLETE 

TYPE 

DISTRO 

PARAM 

1 

PARAM 

2 

CHI 

SQD 

VALUE 

95th 

PERCENTILE 

VALUE 

99th 

PERCENTILE 

VALUE 

mmB. 

0.83 

9  04 

-16.33 

11-20 

Beta 

1.17 

0.56 

4.84 

-24 

-26.86 

21-30 

Beta 

1  43 

0.64 

9:69 

-11.65  . 

-43.42 

31-40 

Beta 

2.05 

0.94 

17.62 

-23.06 

-27.29 

•41.50 

Idgflstic 

*42,63 

10.16 

20.51 

-59.32 

Table  16.  Aircraft  Program  Percentile  Values. 


The  sample  of  aircraft  programs  includes  17  separate  contracts.  Cost  variances 
are  determined  for  each  percentage  completion  bracket.  The  data  are  modeled  using 
BestFit  [Ref.  20],  Appendix  B  contains  complete  information  on  rankings  and  selection  of 
distribution  types.  Percentile  values  were  determined  using  Minitab. 
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2.  The  A-12  Program  Versus  Sample  Aircraft  Programs 

The  reported  A-12  cost  variances  and  percentage  completion  points  were 
compared  to  other  aircraft  programs  to  determine  differences.  The  results  are  provided  in 
Table  17. 


% 

COMPL 

% 

CV 

TYPE 

DISTRO 

AREA 

ABOVE 

VALUE 

7.6 

-7.39 

Beta 

0.736 

11.1 

-11.95 

Beta 

0.7112 

14.9 

-15.94 

Beta 

0.7968 

25.4 

-20.89 

Normal 

0.9997 

31.6 

-24.85 

Beta 

0.9708 

36.8 

-30.78 

Beta 

0.9955 

:  42.1 

-34.09 

Logistic 

0.8921 

46.5 

-38.77 

Logistic 

0.9291 

49  2 

-40.11 

Logistic 

0.9373 

Table  17.  A-12  Versus  Sample  Aircraft  Programs. 

The  distribution  that  best  modeled  the  data  from  the  aircraft  programs  was  used  to 
compute  the  area  under  the  cumulative  density  function  (CDF)  for  the  A-12  values.  This 
was  accomplished  using  Minitab  and  the  CDF  command. 

The  area  given  in  Column  4  is  the  percentage  of  programs  that  the  A-12  cost 
variance  exceeded.  At  7.6  percent  complete,  the  cost  variance  in  the  Program  exceeded 

73.6  percent  of  other  aircraft  programs.  At  termination,  the  A-12  cost  variance  exceeded 

93.7  percent  of  other  aircraft  programs.  The  data  indicate  that  the  A-12  Program  cost 
variances  were  exceptional  when  compared  to  other  aircraft  programs. 

D.  FIXED-PRICE  VERSUS  COST-TYPE  CONTRACTS 
1.  A-12  Cost  Variances  Versus  Contract  Type 

This  analysis  is  conducted  to  determine  the  degree  to  which  the  selection  of  a 
fixed-price  contract  may  have  effected  the  cost  overruns  in  the  A-12  program.  The  mean 
cost  variance  of  a  fixed-price  contract  is  -21.02  percent.  The  median  cost  variance  is 
-10.2  percent.  Fifteen  contracts  were  included  in  the  sample. 
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The  mean  cost  variance  of  cost-type  contracts  is  -14.76  percent  with  a  median 
value  of -6.05  percent.  Thirty-nine  contracts  were  in  the  sample.  Table  18  provides  the 
results  of  a  comparison  between  the  A- 12  cost  variances  and  cost  variances  based  on 
contract  type. 


TYPE 

TYPE 

95th 

PERCENTILE 

99th 

PERCENTILE 

A-12 

AREA 

ABOVE 

SAMPLE 

DISTRO 

VALUE 

VALUE 

VALUE 

A-12 

FP  >  50% 

Logistic 

-47.79 

-51.75 

: -40.11 : 

0628 

CP  >50% 

Beta 

'  -42.73 

-54.52 

-AQAf 

0.9372 

Table  18.  Contract  Type  Comparison. 


The  data  indicate  there  is  a  difference  in  the  cost  variances  based  on  contract  type. 
Compared  to  fixed-price  contracts,  24  percent  of  programs  experienced  worse  cost 
overruns  while  only  5  percent  of  cost-type  programs  experience  worse  overruns. 

This  apparent  difference  was  investigated  using  the  non-parametric  Mann- Whitney 
test.  This  test  determines  if  there  is  statistical  difference  between  the  mean  of  two 
populations.  On  the  basis  of  the  data,  the  test  could  not  reject  the  null  hypothesis  that 
there  was  no  difference,  with  alpha  equal  to  .05.  This  would  refute  the  apparent 
difference  shown  in  Table  18.  Statistically,  there  is  insufficient  evidence  to  claim  there  is  a 
difference  in  the  cost  variances  experienced  by  contract  type. 

2.  Aircraft  Programs  Versus  the  Sample 

A  second  investigation  was  made  to  determine  if  aircraft  programs  experience 
different  overruns  than  the  overall  sample.  The  null  hypothesis  could  not  be  rejected  with 
alpha  equal  to  .05.  There  is  not  sufficient  evidence  to  claim  there  is  a  difference  in  aircraft 
program  cost  variances  and  the  overall  sample. 

E.  ADJUSTMENTS  TO  THE  CONTRACT  BUDGET  BASE 

1.  The  Sample 

All  programs  examined  experienced  adjustments  to  the  CBB.  In  only  two  of  the 
58  contracts  examined  was  there  a  reduction  in  the  CBB.  The  mean  adjustment  to  the 
CBB  is  53  percent  with  a  median  adjustment  of  46  percent. 
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The  effects  of  inflation  were  disregarded  based  on  assumptions  stated  in  Chapter  I 
Hypothesis  4  and  5  were  tested  using  the  Mann- Whitney  test.  For  each  test  alpha  was 
selected  to  be  .05. 

1.  Hypothesis  4.  n cbbfj,  =  pcbbc 

2.  Hypothesis  5.  M.cbbacft  =  |icbbsamplc 

The  alternative  hypothesis  in  each  case  was  the  means  were  not  equal.  For  each 
test,  the  null  hypothesis  could  not  be  rejected  with  rather  strong  evidence.  This  indicates 
there  is  no  statistical  difference  in  the  adjustments  to  the  Contract  Budget  Base  of 
different  types  of  programs  or  different  types  of  contracts.  It  should  be  noted  that  while 
the  means  of  the  CBB  adjustments  appear  to  show  a  wide  difference  the  tests  conducted 

are  based  on  the  variances  of  the  data.  A  summary  of  the  tests  and  criterion  are  provided 
in  Appendix  B. 

2.  A-12  CBB  Adjustments 

Table  19  shows  the  EACs  developed  in  Chapter  IV  for  the  A-12  program,  and  the 
percentage  change  required  to  adjust  the  original  CBB  to  the  EAC.  This  percentage 
adjustment  is  then  compared  to  the  distribution  of  the  sample  adjustments  in  CBB.  A 
percentile  value  is  determined  based  on  the  required  adjustment  to  the  A-12  CBB. 


EAC 

% 

ADJ 

REQD 

TYPE 

DISTRO 

VERSUS 

SAMPLE 

TYPE 

DISTRO 

VERSUS 

SAMPLE 

W/O  AC  FT 

7500 

57 

Logistic 

0.53 

Normal 

0.64 

9069 

89.8 

Logistic 

Q.77 

Normal 

Q.88 

11061 

131.5 

Logistic 

0.93 

Normal 

0,99 

Table  19.  Required  CBB  Adjustments. 


Three  EACs  are  shown  in  Column  1 .  The  first  is  the  PM's  EAC  at  termination  of 
the  Program.  The  second  and  third  EACs  are  the  Rayleigh  derived  estimates  presented 
earlier.  The  percentage  adjustment  that  is  required  to  increase  the  original  CBB  to  the 
estimated  EAC  is  given  in  Column  2.  Column  3  is  the  distribution  best  modeling  the 
distribution  of  CBB  adjustments  for  the  entire  sample.  In  Column  4  is  the  percentage  of 
programs  that  required  a  smaller  percentage  of  adjustment  to  complete  the  EMD  phase. 
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Columns  5  and  6  a  comparison  of  the  A- 12  required  adjustments  versus  the  sample 
without  aircraft  included. 

There  were  not  enough  data  points  for  fixed-price  programs  or  aircraft  programs 
to  provide  a  meaningful  comparison. 

3.  Analysis 

A  CBB  adjustment  to  increase  the  program  funding  to  $7.5  billion  would  require  a 
57  percent  increase  in  the  CBB.  A  57  percent  increase  would  result  in  the  program's 
adjustment  being  greater  than  about  53  percent  of  all  other  programs  in  the  EMD  phase. 

A  CBB  adjustment  to  increase  the  program  funding  to  $9,069  billion  would  have 
required  a  90  percent  increase  in  the  CBB.  This  increase  would  have  exceeded  77  percent 

of  all  other  programs.  The  probability  of  the  A- 12  Program  costing  at  least  $9  billion  is 
.70. 

A  CBB  adjustment  to  increase  the  program  funding  to  $1 1  billion  would  have 
required  a  132  percent  increase  in  the  CBB.  This  would  have  exceeded  93  percent  of  all 
other  programs.  The  probability  of  the  A- 12  Program  costing  at  least  $1 1  billion  is  .49. 

F.  COST  VARIANCES  OF  EMD  CONTRACTS 
1.  Results  of  the  Research 


This  research  examined  fifty-eight  total  contracts.  Table  20  summarizes  the 
findings  on  cost  variances  in  the  EMD  phase  of  the  acquisition  process. 


SAMPLE 

MEAN 

CV% 

MEDIAN 

CV% 

99th 

PERCENTILE 

N 

FIXED  PRICE 

-21.02 

-10.2 

-94.81 

15 

COST  TYPE 

-1476 

-6.05 

-51.77 

39 

AIRCRAFT 

-14.03 

-6.13 

-49.18 

15 

TOTAL 

-13.63 

-6.09 

-51.26 

40 

Table  20.  EMD  Cost  Variances. 


2.  Findings 

Fixed-price  contracts  tend  to  have  the  highest  cost  variance  of  either  contract  type. 
The  mean  cost  variance  for  fixed-price  contracts  is  -21.02  percent.  Cost-type  contracts 
require  the  greatest  CBB  adjustment,  57.24  percent,  compared  to  fixed-price  contracts 
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that  require  a  43  percent  increase.  Aircraft  programs,  in  general,  do  not  have  exceptional 
cost  variances,  but  they  do  require  above  average  CBB  adjustments. 

G.  SUMMARY 

This  chapter  examined  the  data  from  the  sample  58  contracts  and  compared  the 
results  to  the  A- 12  Program.  Compared  to  the  sample  contracts,  the  overruns  in  the  A- 12 
program,  at  41  percent  over  programmed  budget,  were  exceptional.  To  increase  the 
funding  for  the  program  would  have  required  an  increase  of  approximately  $5  to  $7 
billion.  This  amount  of  CBB  adjustment  would  have  been  exceptional  as  well. 

The  final  chapter.  Chapter  VI,  reviews  the  research  questions  and  the  findings. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  REVIEW  OF  THE  RESEARCH  QUESTIONS 

1.  The  Primary  Research  Question 

Were  cost  overruns  in  the  A- 12  Program  exceptional? 

The  answer  to  this  question  was  intended  to  be  a  simple  yes  or  no.  In  keeping 
with  this  intent  the  answer  is  yes.  The  cost  variances  in  the  A- 12  program  were 
exceptional  when  compared  to  the  sample  contracts  in  the  EMD  phase. 

The  justification  for  this  finding  is  based  on  a  comparison  of  the  percentile  values 
of  the  A- 12  cost  variances,  measured  against  other  programs  at  similar  points  of 
completion.  Table  6,  in  Chapter  IV,  shows  the  cost  variances  of  the  A- 12.  Table  15  in 
Chapter  V,  provides  the  percentile  values  for  the  A- 12  and  the  area  of  the  distribution  that 
exceeded  the  point  values  of  the  cost  variances  for  the  A-12.  A  test  to  see  if  the  cost 
variances  of  the  A-12  were  no  different  than  other  aircraft  programs  was  conducted  and 
the  same  results  were  found.  The  results  of  the  comparison  to  other  aircraft  can  be  found 
in  Table  17,  in  Chapter  V. 

The  A-12  cost  variances  clearly  exceeded  a  majority  of  other  program's  cost 
variances.  Compared  to  the  sample  of  58  contracts,  the  cost  overrun  of  the  A-12 
exceeded  91  percent  of  the  others.  At  termination,  the  A-12  exceeded  94  percent  of  other 
aircraft  program's  cost  overruns.  To  ensure  these  results  were  not  biased  by  the  selection 
of  the  distribution,  at  least  one  different  distribution  was  applied  in  each  case  to  determine 
if  like  results  would  be  obtained.  In  each  case  the  cost  variance's  percentile  value 
remained  extreme.  The  comparison  distribution's  percentile  values  can  be  found  in 
Appendix  B. 

Measurements  of  only  the  cost  variances  do  not  account  for  adjustments  to  the 
Contract  Budget  Base.  A  program  with  a  large  cost  variance  can  be  rebaselined  to 
eliminate  the  variance.  Ignoring  the  CBB  adjustments  would  bias  the  findings.  The 
impact  of  CBB  adjustments  was  tested  in  the  subsidiary  research  questions. 
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2.  Subsidiary  Research  Questions 

What  would  the  A-12  FSD  program  have  cost  if  it  continued  to  completion? 

It  is  likely  that  the  program  would  have  cost  between  $9  and  $1 1  billion.  Based  on 
a  Rayleigh  model  estimate  at  completion  and  the  MMAE  methodology  developed  by 
Gallagher,  the  costs  and  probabilities  for  the  estimates  at  completion  are  as  follows: 

a.  Given  a  June  1992,  first  flight  date  the  program  would  have  been 
completed  in  September  1999.  The  program  would  have  cost  $9  billion.  The  cumulative 
probability  of  the  program  exceeding  this  cost  is  .63. 

b.  Given  a  December  1992,  first  flight  date  the  program  would  have  been 
completed  in  January  2001 .  The  program  would  have  cost  $1 1  billion.  The  cumulative 
probability  of  the  program  exceeding  this  cost  is  .49. 

These  results  are  consistent  with  the  DoD  IG  estimates  contained  in  Table  3,  in 
Chapter  III.  The  DoD  IG  estimates  were  prepared  by  the  Cost  Analysis  Improvement 
Group. 

The  budget  adjustments  that  would  have  been  required  to  finish  the  program 
would  have  exceeded  between  77  and  93  percent  of  all  other  programs.  To  eliminate  the 
cost  variance  would  have  required  an  above  average  budget  adjustment. 

Was  the  Secretary  of  Defense  justified  in  terminating  the  Program  for  cost 
overruns? 

Yes.  The  cost  variances  in  the  A-12  Program  exceeded  97  percent  of  all  other 
programs,  as  shown  in  Table  15,  in  Chapter  V.  A  budget  adjustment  between  89  and  132 
percent  would  have  been  required  to  finish  the  program.  While  this  amount  of  adjustment 
to  the  budget  would  be  within  the  distribution  of  CBB  adjustments,  (See  Table  19)  it 
would  have  as  a  minimum  exceeded  77  percent  of  all  other  Contract  Budget  Base 
adjustments. 

Are  there  differences  in  cost  overruns  between  aircraft  programs  and  other 
types  of  EMD  programs? 

No.  There  is  insufficient  evidence  to  conclude  that  aircraft  programs  experience 
significantly  different  cost  variances.  The  test  results  could  not  reject  the  null  hypothesis 
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that  there  were  no  differences.  The  test  showed  there  was  no  statistical  difference  with 
alpha  equal  to  .05. 

Are  there  differences  in  cost  overruns  between  fixed-price  contracts  and  cost- 
type  contracts  for  developmental  work? 

No.  There  is  not  sufficient  evidence  to  claim  that  fixed-price  contracts  experience 
statistically  different  overruns.  As  measured  by  the  Mann- Whitney  non-parametric  test, 
the  null  hypothesis  could  not  be  rejected  with  alpha  equal  to  .05. 

What  is  the  average  cost  overrun  of  an  EMD  contract? 

The  mean  cost  overrun  for  an  EMD  contract  is  13.37  percent,  as  shown  in  Table 
20,  in  Chapter  V.  The  mean  cost  overrun  for  aircraft  programs  is  14.03  percent.  The 
mean  cost  overrun  for  fixed-price  contracts  is  21.02  percent  and  the  mean  overrun  for 
cost-type  contracts  is  14.76  percent.  The  research  found  that  the  mean  value  for  overruns 
tends  to  overstate  the  extent  of  the  problem.  The  use  of  medians  provides  a  more 
accurate  picture.  The  median  cost  overruns  for  the  above  contracts  ranged  from  6.05  to 
10.2  percent. 

To  examine  the  impact  of  rebaselining  the  percentage  change  in  each  contract's 
CBB  was  determined.  The  results  of  this  examination  are  provided  in  Table  7,  in  Chapter 
IV.  To  eliminate  the  possibility  that  a  normal  rebaseline  would  have  prevented  the 
extraordinary  overruns  in  the  A-12  program,  the  required  adjustment  to  the  A-12's  CBB 
was  examined. 

Depending  on  the  date  of  first  flight  (assuming  it  would  be  no  later  than  December 
of  1992),  the  required  CBB  adjustment  is  between  89  and  132  percent.  An  adjustment  of 
this  magnitude  exceeds  approximately  77  to  91  percent  of  all  other  programs.  The 
required  mean  adjustments  were  found  to  vary  dependent  on  program  and  contract  type. 
The  results  of  this  investigation  are  provided  in  Table  19,  in  Chapter  V. 

There  is  insufficient  evidence  to  believe  that  the  adjustments  to  the  CBB  vary 
depending  on  either  program  type  or  contract  type.  The  apparent  difference  in  mean 
values  for  the  contract  types  was  investigated  using  the  Mann-Whitney  test.  The  test 
showed  that  the  null  hypothesis  could  not  be  rejected. 
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B.  CONCLUSIONS 

1.  The  Decision  to  Terminate  the  Program 

The  decision  of  the  Secretary  of  Defense  to  terminate  the  A- 12  Program  was 
justified  The  A- 12  Program  experienced  significantly  different  cost  variances  from  other 
programs  examined.  Hypothesis  1,  the  A- 12  overruns  were  no  different  than  other 
programs,  was  tested  to  determine  if  A- 12  cost  overruns  were  the  same  as  other  EMD 
contracts.  This  was  not  proven.  At  termination  the  A- 12  Program  was  41  percent  overrun 
from  its  baseline.  This  amount  of  overrun  exceeded  97  percent  of  other  programs  in  the 
sample.  The  test  compared  percentile  values  of  the  A- 12  versus  the  sample. 

2.  Fixed-Price  Contracts  and  Cost  Overruns 

The  assertion  that  the  cause  of  the  A-12's  failure  was  due  to  the  use  of  a 
fixed-price  contract  can  not  be  supported  by  the  data.  The  selection  of  contract  type  is  not 
a  factor  in  preventing  cost  overruns.  The  fixed-price  contracts  in  the  sample  had  a  larger 
overrun  than  cost-type  contracts,  21.02  percent  versus  14.76  percent.  The  median 
overrun  values  were  much  closer:  10.2  versus  6.05  percent.  Fixed-price  contracts  required 
smaller  budget  adjustments  than  cost  contracts.  Hypothesis  2  was  tested  to  see  if  the 
overruns  are  the  same.  The  null  hypothesis,  that  there  was  no  difference  between  cost 
overruns  in  fixed-price  contracts  and  cost  contracts,  could  not  be  rejected. 

3.  Aircraft  Program  Cost  Variances 

The  mean  cost  variance  of  aircraft  programs  is  the  same  as  other  types  of 
programs.  This  indicates  that  the  cost  overruns  in  an  aircraft  program  will  be  no  higher 
than  any  other  type  of  program.  Hypothesis  3,  was  tested  for  differences  in  the  mean 
overruns  of  aircraft  programs  and  other  types  of  programs.  The  null  hypothesis  could  not 
be  rejected,  no  significant  difference  exists.  The  median  overruns  of  the  sample  examined 
were  6. 13  percent  for  aircraft  and  6.09  percent  for  other  types  of  programs. 

4.  Budget  Adjustments  for  Fixed-Price  and  Cost-Type  Contracts 

The  required  budget  adjustments  for  fixed-price  and  cost-type  contracts  are  the 
same.  The  budget  adjustment  is  the  amount,  measured  by  percentage,  required  to  adjust 
the  initially  specified  budget  to  the  ending  budget.  In  other  words,  it  is  the  amount  of 
additional  money  the  program  manager  needs  to  finish  the  program.  The  budget 
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adjustment  required  to  finish  the  A- 12  Program  would  have  been  between  a  77  and  93 
percent  increase.  The  mean  budget  adjustment  for  fixed-price  contracts  is  44  percent  and 
the  mean  budget  adjustment  for  cost-type  contracts  is  57  percent.  Hypothesis  4  was 
tested  to  see  if  there  are  differences  between  the  Contract  Budget  Base  adjustments  for 
fixed-price  contracts  and  the  CBB  adjustments  for  cost-type  contracts. 

A  visual  inspection  of  the  data  would  indicate  a  difference,  but  the  Mann- Whitney 
test  showed  no  statistical  difference.  This  can  be  accounted  for  by  the  dispersion  of  the 
data.  Prior  to  conducting  the  test  the  data  was  normalized  by  dividing  each  data  point  by 
its  standard  deviation.  The  null  hypothesis,  that  there  are  no  differences,  could  not  be 
rejected. 

5.  Budget  Adjustments  for  Aircraft  Programs 

Aircraft  programs  require  about  the  same  amount  of  budget  adjustment  as  other 
types  of  programs.  The  mean  budget  adjustment  for  aircraft  programs  was  higher  than 
other  types  of  programs.  The  dispersion  of  the  values  was  much  greater  for  aircraft 
programs  than  for  other  types  of  programs,  and  the  data  were  normalized  prior  to  testing. 
Hypothesis  5  was  tested  to  determine  if  aircraft  programs  experience  greater  mean  CBB 
adjustments  than  other  programs.  The  null  hypothesis  could  not  be  rejected.  There  is  no 
statistical  difference  in  the  mean  budget  adjustments  of  aircraft  programs  and  the  sample. 

6.  Mean  Values  Versus  Median  Values 

Mean  values  overstate  cost  overruns.  To  compare  a  program's  overruns  with 
other  programs,  the  median  cost  overrun  should  be  used.  The  mean  is  a  good  measure  of 
central  tendency  for  data  modeled  by  the  Normal  distribution.  Cost  overruns  are  modeled 
by  the  Beta  distribution.  The  use  of  the  median  is  a  more  accurate  measure  of  central 
tendency  for  skewed  data.  As  shown  in  Figure  3,  in  Chapter  II,  the  distribution  of  cost 
overruns  is  skewed  to  the  left.  The  use  of  mean  values  can  be  misleading.  For  example, 
the  mean  cost  variance  of  fixed-price  contracts  and  cost-type  contracts  is  -21.02  percent 
and  -14.76  percent  respectively.  At  first  glance  the  conclusion  could  be  drawn  that 
fixed-price  contracts  are  significantly  different  from  cost-type  contracts. 

The  median  values  for  the  cost  variances  of  fixed-price  and  cost-type  contracts  are 
-10.2  percent  and  -6.05  percent  respectively.  The  median  values  show  less  difference  by 
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contract  type.  The  test  of  the  hypothesis  found  no  statistical  difference  in  cost  variances 
based  on  contract  type. 

7.  Cost  Variances  of  EMD  Contracts 

The  results  of  this  research  agree  with  prior  research,  cost  overruns  do  not  get 
better  with  time.  Programs  less  than  10  percent  complete  have  a  positive  cost  variance 
(cost  underrun).  From  1 1  percent  complete  on  programs  tend  to  experience  cost 
overruns.  From  1 1  to  80  percent  complete  the  cost  overrun  increases  with  program 
length.  The  largest  cost  overrun  for  the  sample,  was  from  71  to  80  percent  complete  and 
was  negative  14.52  percent.  The  mean  value  of  the  cost  variance  of  the  sample  was 
negative  13.37  percent  while  the  median  value  was  negative  6.09  percent.  Very  few 
contracts  were  found  to  have  positive  cost  variances  at  any  point  in  the  program.  A 
summary  of  the  findings  on  cost  variances  is  provided  in  Table  1 1,  Chapter  IV,  and  Table 
20  in  Chapter  V. 

The  observations  of  Abba  and  Christie  and  the  empirical  findings  of  Heise  were 
confirmed  in  that  the  cost  variances  tended  to  worsen  as  the  program  progressed.  This  is 
shown  in  Table  1 1,  in  Chapter  IV. 

8.  Causes  of  Cost  Growth 

The  cause  of  the  A-12's  cost  overruns  are  difficult  to  determine.  It  was  not  the 
intent  of  this  thesis  to  provide  the  answer  to  why  the  A- 12  had  exceptional  overruns. 
Based  on  the  data  examined,  it  appears  that  the  Program  encountered  significantly  greater 
technical  challenges  than  were  anticipated,  which  delayed  first  flight  and  increased  cost. 

The  contracts  examined  in  this  research  all  showed  variances  from  the  baseline. 
The  majority  of  these  contracts  experienced  cost  overruns  throughout  the  program.  The 
Rand  report  [Ref.  1 0]  found  no  single  cause  for  the  cost  growth  of  weapon  system 
programs.  The  report  found  a  correlation  between  the  size  of  the  program  and  the  length 
of  the  program  and  cost  growth.  The  smaller  the  program,  the  greater  the  variance  and 
the  longer  the  program  is  carried  out,  the  greater  the  variance. 

A  1992  GAO  report  summarizes  the  problem  of  determining  the  cause  of  cost 
growth: 
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The  weapon  system  problems  we  have  reported  in  the  past  2  years 
mirror  those  we  reported  in  the  1970s  and  the  1980s.  This  does  not  imply 
that  the  history  of  weapon  acquisitions  amounts  to  a  string  of  bad 
programs.  The  point  is  that,  despite  conscious  attempts  to  improve  the 
acquisition  process,  weapons  still  cost  more,  take  longer  to  field,  often 
encounter  performance  problems,  and,  in  many  instances  are  difficult  to 
produce  or  support.  The  persistence  of  these  problems  reflects  the  fact 
that  the  design,  development,  and  production  of  major  weapon  systems  are 
extremely  complex  technical  processes.... In  short,  it  takes  a  myriad  of 
things  to  go  right  for  a  program  to  be  successful,  but  only  a  few  things  to 
go  wrong  to  cause  major  problems.  [Ref.  27:  p.  15] 

C.  RECOMMENDATIONS 

1.  The  A-12 

The  Government's  decision  to  terminate  the  fixed-price  contract  of  the  A-12  is 
justified,  based  on  an  examination  of  the  data  used  in  this  research,  The  Government 
should  continue  to  pursue  the  case  in  court  and  demand  reimbursement  for  the 
unliquidated  progress  payments. 

2.  Cost  Estimating 

The  findings  of  this  research  indicate  that  estimates  at  completion  in  EMD 
contracts  are  too  optimistic.  As  shown  in  Table  7,  the  mean  adjustment  to  the  CBB  is  54 
percent.  Only  two  of  the  58  contracts  showed  downward  revisions  in  the  CBB.  More 
emphasis  should  be  placed  on  developing  realistic  estimates  of  the  cost  of  the  contract.  A 
cost  underrun  in  the  early  stages  of  a  program  is  a  poor  indicator  of  future  cost 
performance.  Fully  defining  the  scope  of  work  before  the  start  of  EMD  may  help  alleviate 
the  problem.  A  longer  Demonstration  and  Validation  phase  may  be  warranted  in 
programs  with  substantial  technical  risk. 

3.  Fixed-Price  Contracts  for  EMD 

The  belief  that  fixed-price  contracts  place  the  majority  of  the  cost  risk  on  the 
contractor  is  not  supported  by  the  data  for  EMD  contracts.  No  statistical  difference  was 
shown  in  the  median  values  for  cost-type  contracts  and  fixed-price  contracts.  While 
fixed-price  contracts  showed  more  extreme  points  than  cost-type  contracts,  this  may  have 
been  due  to  the  Government's  reluctance  to  adjust  the  price  of  the  fixed-price  type 
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contracts.  As  shown  in  Table  10  the  Air  Force  has  used  a  fixed-price  type  contract  in  the 
majority  of  EMD  contracts  that  were  examined. 

The  decision  to  use  a  cost-type  contract  will  not  have  any  impact  on  the  cost 
variance  of  the  contract.  The  mean  budget  adjustment  for  fixed-price  contracts  was  44 
percent,  the  median  value  was  54  percent,  while  the  mean  budget  adjustment  for  cost-type 
contracts  was  57  percent  and  the  median  value  was  45  percent.  By  examining  the  means 
for  budget  adjustments  the  conclusion  could  be  drawn  that  the  use  of  a  fixed-price 
contract  would  require  a  smaller  budget  adjustment.  The  use  of  median  values  would 
show  that  the  use  of  a  fixed-price  contract  would  require  a  greater  budget  adjustment. 

The  mean  values  can  be  skewed  by  outliers,  extreme  values  by  very  few  programs. 

D.  AREAS  FOR  FURTHER  RESEARCH 

1.  The  Rayleigh  Distribution  Model 

The  Rayleigh  model  shows  promise  as  a  cost  estimating  tool.  Further  research 
into  optimizing  this  model  for  more  narrowly  defined  programs  should  be  examined. 
Adjustment  of  the  parameters  of  the  model  resulted  in  greater  accuracy.  This  research 
used  only  ten  values  to  optimize  the  parameters  for  aircraft  contracts.  By  using  more 
values  to  optimize  the  parameters  an  even  more  refined  model  could  be  achieved. 

The  Rayleigh  model  proved  to  be  a  better  estimator  of  costs  at  completion  than  the 
CPI  techniques  given  an  unstable  baseline.  In  programs  where  the  baseline  is  relatively 
stable  the  CPI  technique  is  both  easy  to  use  and  accurate.  The  advantage  that  the 
Rayleigh  technique  provides  is  the  ability  to  provide  estimates  independent  of  the  baseline. 

It  was  not  the  intent  of  this  research  to  prove  the  value  of  the  Rayleigh  model. 

The  model  was  selected  based  on  its  ability  to  provide  EACs  independent  of  current 
estimates  of  final  costs.  Other  techniques  exist  for  estimating  costs  that  were  also  not 
examined. 

2.  Fixed-Price  Contracts 

This  research  indicates  that  the  use  of  fixed-price  contracts  in  developmental  work 
is  not  inherently  bad.  All  contracts  examined  had  adjustments  made  to  the  CBB  regardless 
of  contract  type.  Additional  work  could  examine  in  more  detail  the  differences  in  cost 
variances  between  cost-type  contracts  and  fixed-price  type  contracts. 
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APPENDIX  A.  CONTRACT  DATABASE 


This  appendix  contains  the  contracts  used  in  the  research.  The  agreement  with 
OUSD  (A&T)  requires  that  the  name  of  the  contract  and  the  contractor  not  be  disclosed. 
The  contracts  are  listed  by  number.  The  corresponding  list  of  contracts  can  be  released 
with  the  permission  of  the  OUSD  (A&T).  The  list  of  contracts  used  in  this  research  is 
maintained  by  Dr.  David  V.  Lamm,  Systems  Management  Department,  Naval 
Postgraduate  School. 

ABBREVIATIONS  USED 

1 .  ACWP.  The  reported  actual  cost  of  work  performed  drawn  from  the  database.  13 . 

2.  ALLOC  BUDGET.  Actual  budget  committed  to  the  program. 

3.  BCWP.  The  reported  budgeted  cost  of  work  performed  drawn  from  the  database. 

4.  CBB.  Contract  Budget  Base. 

5.  COMPL  PT.  Percentage  completion  point.  As  measured  by  BCWP/ ALLOC 

BUDGET 

6.  CP.  Cost  Plus. 

7.  CPAF.  Cost-Plus-Award-Fee. 

8.  CPIF.  Cost-Plus-Incentive-Fee. 

9.  CV.  Cost  Variance.  Percentage  cost  variance  determined  by  (BCWP-ACWP)/BCWP. 

10.  FP.  Firm-Fixed-Price 

11.  FPIF.  Fixed-Price-Incentive-Firm. 

12.  FSD.  Full-Scale  Development.  Equivalent  to  Engineering  and  Manufacturing 
Development  (EMD).  Drawn  from  the  description  in  the  DAES  database.  "DEV"  as 
used  in  the  DAES  database  is  equivalent  to  EMD. 

13.  K.  Contract. 

14.  KTYPE.  Contract  type. 

15.  SYSTEM.  The  type  of  system  the  contract  was  for.  General  categories  were 
established. 
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SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

AF 

MISSILE 

FSD 

ALLOC 

FPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

455.3 

663.7 

514.2 

514.2 

88.55% 

-45.77% 

460.3 

677.7 

530.3 

530.3 

86.80% 

-47.23% 

465.8 

688.3 

539.3 

539.3 

86.37% 

-47.77% 

476.5 

707.1 

562.1 

562.1 

84.77% 

-48.39% 

495.5 

718.3 

560.4 

560.4 

88.42% 

-44.96% 

724.6 

726.3 

606.3 

833 

86.99% 

-0.23% 

747.5 

748.2 

595.4 

822.1 

90.93% 

-0.09% 

760.4 

759.9 

596.9 

823.6 

92.33% 

0.07% 

766 

771.9 

591.8 

814.1 

94.09% 

-0.77% 

SERVICE: 

ARMY 


SYSTEM: 

ELECTR 


PHASE: 

DEV 


ALLOC 

BUDGET 


K  TYPE: 
CPAF 

COMPL  PT 


11.5 

11.8 

54.5 

54.5 

21.10% 

-2.61  % 

16.1 

16.5 

61.1 

61.1 

26.35% 

-2.48% 

21 .6 

22.2 

67.9 

67.9 

31.81% 

-2.78% 

28.7 

30.7 

68.6 

68.6 

41 .84% 

-6.97% 

35.6 

39.1 

68.7 

68.7 

51.82% 

-9.83% 

42.3 

46.9 

65.5 

65.5 

64.58% 

-10.87% 

49 

52.3 

66 

66 

74.24% 

-6.73% 

55.7 

60.7 

66.4 

66.4 

83.89% 

-8.98% 

60.2 

68.1 

68.5 

68.5 

87.88% 

-13.12% 

63.3 

73.7 

69.4 

69.4 

91.21% 

-16.43% 

66.3 

78.5 

71.5 

71.5 

92.73% 

-18.40% 

68.7 

84.8 

73.4 

73.4 

93.60% 

-23.44% 

70.8 

90 

75 

75 

94.40% 

-27.12% 

72 

94.4 

75 

75 

96.00% 

-31.11% 

71 

96.1 

73 

73 

97.26% 

-35.35% 

71.7 

98.9 

72.9 

72.9 

98.35% 

-37.94% 

K#: 

3 

SERVICE: 

AF 

BCWP 

SYSTEM: 

MISSILE 

ACWP 

PHASE: 

FSD 

CBB 

ALLOC 

BUDGET 

K  TYPE: 

FPIF 

COMPL  PT 

CV 

10.7 

10.9 

376.1 

376.1 

2.84% 

-1.87% 

37.9 

38.9 

386.6 

386.6 

9.80% 

-2.64% 

56.9 

58.4 

386.6 

386.6 

14.72% 

-2.64% 

85.1 

89.4 

386.7 

386.7 

22.01% 

-5.05% 

97.1 

101.7 

395.1 

395.1 

24.58% 

-4.74% 

118.9 

133.1 

395.1 

395.1 

30.09% 

-11.94% 

161.2 

193.6 

395.1 

395.1 

40.80% 

-20.10% 

202.4 

257.4 

395.1 

395.1 

51 .23% 

-27.17% 

220.5 

285.4 

395.1 

395.1 

55.81% 

-29.43% 

246.9 

335.4 

395.3 

395.3 

62.46% 

-35.84% 

273.6 

388.7 

396.4 

396.4 

69.02% 

-42.07% 

303.7 

459.7 

394.6 

394.6 

76.96% 

-51.37% 

307.7 

473.8 

394.6 

394.6 

77.98% 

-53.98% 

319.4 

517 

394 

394 

81.07% 

-61 .87% 

336.6 

589 

394 

394 

85.43% 

-74.99% 

355.7 

644.6 

394 

394 

90.28% 

-81.22% 

344 

725.1 

394 

807 

42.63% 

-110.78% 

351.2 

726.2 

392.7 

807 

43.52% 

-106.78% 

354.5 

731.9 

392.7 

807 

43.93% 

-106.46% 

359.6 

739.5 

393.6 

807 

44.56% 

-105.65% 

362.5 

746.8 

393.6 

807 

44.92% 

-106.01% 

365.6 

753.5 

393.6 

807 

45.30% 

-106.10% 

369 

761 

393.6 

807 

45.72% 

-106.23% 

371.3 

767.2 

393.6 

807 

46.01% 

-106.63% 

372.6 

770.8 

393.6 

807 

46.17% 

-106.87% 

j 

t 

374.3 

781 .3 

393.6 

807 

46.38% 

-108.74% 
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SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

AF 

MISSILE 

FSD 

ALLOC 

FPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

2.4 

2.4 

15.1 

15.1 

15.89% 

0.00% 

3 

3 

15.1 

15.1 

19.87% 

0.00% 

3.9 

4 

15.1 

15.1 

25.83% 

-2.56% 

5.1 

5.1 

15.1 

15.1 

33.77% 

0.00% 

7.2 

7.2 

15.1 

15.1 

47.68% 

0.00% 

8.5 

8.4 

15.1 

15.1 

56.29% 

1.18% 

9.5 

9.7 

15.1 

15.1 

62.91% 

-2.11% 

11.9 

12.5 

15.1 

15.1 

78.81% 

-5.04% 

12.8 

14.4 

23.5 

23.5 

54.47% 

-12.50% 

14.2 

15.4 

23.5 

23.5 

60.43% 

-8.45% 

15.8 

16.4 

23.5 

23.5 

67.23% 

-3.80% 

16.2 

16.7 

23.5 

23.5 

68.94% 

-3.09% 

16.6 

16.9 

23.5 

23.5 

70.64% 

-1.81% 

17 

17.2 

23.5 

23.5 

72.34% 

-1.18% 

17.4 

17.6 

23.5 

23.5 

74.04% 

-1.15% 

18 

18.1 

23.5 

23.5 

76.60% 

-0.56% 

18.4 

18.4 

23.5 

23.5 

78.30% 

0.00% 

19.2 

19.7 

23.5 

23.5 

81 .70% 

-2.60% 

20.5 

21.5 

23.5 

23.5 

87.23% 

-4.88% 

SERVICE:  SYSTEM:  PHASE: 

NAVY  ELECTRONICS  FSD 


ALLOC 

BUDGET 


K  TYPE: 
CPIF 

COMPL  PT 


12.65% 

21.66% 


13.20% 

15.96% 

32.43% 

36.24% 

41.15% 

35.10% 

35.71% 

40.38% 

41.36% 

62.46% 

69.69% 

73.58% 

77.87% 

79.28% 

82.22% 

84.06% 

85.15% 

88.48% 

90.48% 

90.86% 


-3.16% 
-3.62% 
-5.40% 
-11.29% 
-12.58% 
-16.48% 
-17.25% 
-18.30% 
-1.34% 
-0.64% 
-2.03% 
-1.85% 
-2.19% 
-2.84% 
-4.08% 
-5.32% 
-5.79% 
-5.88% 
-4.63% 
-5  23% 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

6 

ARMY 

AIRCRAFT 

DEV 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

0.10 

2.6 

293.9 

293.9 

0.03% 

-25.00 

0.20 

7.4 

293.9 

293.9 

0.07% 

NA 

18.9 

23.1 

293.9 

293.9 

6.43% 

-22.22% 

30 

39.5 

293.9 

293.9 

10.21% 

-31.67% 

56.3 

62.7 

292.9 

292.9 

19.22% 

-11.37% 

87.9 

90.4 

348.5 

348.5 

25.22% 

-2.84% 

112.5 

118.4 

366.1 

366.1 

30.73% 

-5.24% 

136.7 

141.5 

368 

368 

37.15% 

-3.51% 

160.9 

169.2 

369.6 

369.6 

43.53% 

-5.16% 

185.8 

196.2 

369.8 

369.8 

50.24% 

-5.60% 

209.6 

225.6 

374 

374 

56.04% 

-7.63% 

233.3 

255.4 

374.5 

374.5 

62.30% 

-9.47% 

259 

288.4 

374 

374 

69.25% 

-11.35% 

280.3 

324 

382.9 

382.9 

73.20% 

-15.59% 

295.7 

347.4 

383.3 

383.3 

77.15% 

-17.48% 

324.9 

384.2 

379 

379 

85.73% 

-18.25% 

311.6 

404.2 

382.1 

382.1 

81.55% 

-29.72% 

340.3 

435.3 

386.1 

386.1 

88.14% 

-27.92% 

350.1 

458.4 

422.7 

422.7 

82.82% 

-30.93% 

370.4 

489.5 

422.7 

422.7 

87.63% 

-32.15% 

396.4 

527.8 

423.4 

423.4 

93.62% 

-33.15% 

414.2 

545.6 

428.2 

428.2 

96.73% 

-31.72% 

418.9 

554.1 

429.5 

429.5 

97.53% 

-32.28% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

7 

ARMY 

AIRCRAFT 

DEV 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

0.20 

0.1 

27.7 

27.7 

0.72% 

0.50 

1.10 

1 

27.7 

27.7 

3.97% 

9.09% 

3.7 

3.6 

27.7 

27.7 

13.36% 

2.70% 

7.1 

6.7 

27.7 

27.7 

25.63% 

5.63% 

9.7 

9.9 

27.7 

27.7 

35.02% 

-2.06% 

12.3 

12.3 

27.7 

27.7 

44.40% 

0.00% 

13.3 

13.4 

27.7 

27.7 

48.01% 

-0.75% 

15.1 

15 

27.7 

27.7 

54.51% 

0.66% 

15.9 

15.9 

37.4 

37.4 

42.51% 

0.00% 

16.8 

17 

37.4 

37.4 

44.92% 

-1.19% 

17.9 

18.1 

37.4 

37.4 

47.86% 

-1.12% 

19.2 

19.6 

37.4 

37.4 

51 .34% 

-2.08% 

20.6 

21.2 

37.4 

37.4 

55.08% 

-2.91% 

22.4 

23 

37.4 

37.4 

59.89% 

-2.68% 

24.6 

24.7 

37.4 

37.4 

65.78% 

-0.41% 

26.2 

26.5 

37.4 

37.4 

70.05% 

-1.15% 

26.7 

27.9 

37.5 

37.5 

71 .20% 

-4.49% 

29.8 

30.5 

37.6 

37.6 

79.26% 

-2.35% 

32.6 

33.1 

37.6 

37.6 

86.70% 

-1.53% 

34.4 

34.6 

37.6 

37.6 

91 .49% 

-0.58% 

36.2 

36.6 

37.6 

37.6 

96.28% 

-1.10% 

38 

38.2 

37.6 

37.6 

101.06% 

-0.53% 

69 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

8 

NAVY 

AIRCRAFT 

FSD 

ALLOC 

CP 

. . . 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

362.10 

369.1 

582.3 

582.3 

62.18% 

-0.02 

407.30 

412 

586.5 

586.5 

69.45% 

-1.15% 

419.7 

430.3 

582.8 

582.8 

72.01% 

-2.53% 

440.2 

453 

582.8 

582.8 

75.53% 

-2.91% 

453.9 

466.1 

582.8 

582.8 

77.88% 

-2.69% 

480.4 

491.9 

582.8 

582.8 

82.43% 

-2.39% 

489.9 

502.6 

583.7 

583.7 

83.93% 

-2.59% 

505.8 

518.6 

584 

584 

86.61% 

-2.53% 

522 

540.3 

584 

584 

89.38% 

-3.51% 

527.1 

528.4 

584 

584 

90.26% 

-0.25% 

541.5 

554.3 

586 

586 

92.41% 

-2.36% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

9 

NAVY 

ELECTRONICS 

DEV 

ALLOC 

CPFF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

533.10 

536.6 

846.4 

846.4 

62.98% 

-0.01 

649.10 

650.8 

887 

887 

73.18% 

-0.26% 

676 

673.9 

869.9 

869.9 

77.71% 

0.31% 

711.3 

709.5 

873.5 

873.5 

81.43% 

0.25% 

738.9 

739.3 

886 

886 

83.40% 

-0.05% 

762.7 

765.3 

891.2 

891.2 

85.58% 

-0.34% 

775.9 

780.5 

895.4 

895.4 

86.65% 

-0.59% 

802.7 

802.3 

942.7 

942.7 

85.15% 

0.05% 

822.2 

823.6 

941.6 

941.6 

87.32% 

-0.17% 

840.4 

839.6 

942.6 

942.6 

89.16% 

0.10% 

851.1 

849.3 

944.6 

944.6 

90.10% 

0.21% 

875.5 

872.4 

920.6 

920.6 

95.10% 

0.35% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

10 

AF 

AIRCRAFT 

DEV 

ALLOC 

CP 

... 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

1172.80 

1175.7 

1516.9 

1516.9 

77.32% 

-0.00 

1227.80 

1232.7 

1518.7 

1518.7 

80.85% 

-0.40% 

1290.1 

1293.7 

1809.7 

1809.7 

71 .29% 

-0.28% 

1356.3 

1353.1 

1846.3 

1846.3 

73.46% 

0.24% 

1442.5 

1437.3 

1898.8 

1898.8 

75.97% 

0.36% 

1466.1 

1461.4 

1915.8 

1915.8 

76.53% 

0.32% 

1523.2 

1540.4 

2010.6 

2035.8 

74.82% 

-1.13% 

1598.4 

1626.7 

2034.1 

2059.3 

77.62% 

-1.77% 

1667.1 

1713.2 

2049.1 

2074.3 

80.37% 

-2.77% 

1738.8 

1800.3 

2069.5 

2094.7 

83.01% 

-3.54% 

1769.4 

1834.2 

2118 

2143.2 

82.56% 

-3.66% 

1889.6 

1974.3 

2119.6 

2146 

88.05% 

-4.48% 

1973 

2032.6 

2156.8 

2223.1 

88.75% 

-3.02% 

2028.1 

2084.1 

2178.4 

2244.6 

90.35% 

-2.76% 

2063.7 

2114.2 

2173.1 

2239.3 

92.16% 

-2.45% 

2115.3 

2139.4 

2182.8 

2226.2 

95.02% 

-1.14% 

70 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

11 

NAVY 

MISSILE 

FSD 

ALLOC 

CP 

... 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

36.50 

38.2 

83.7 

83.7 

43.61% 

-0.05 

48.50 

52.8 

120.3 

120.3 

40.32% 

-8.87% 

57 

62 

119.9 

119.9 

47.54% 

-8.77% 

81.1 

85.5 

124.8 

124.8 

64.98% 

-5.43% 

65.5 

69.2 

102.1 

102.1 

64.15% 

-5.65% 

85.3 

91.5 

111 

111 

76.85% 

-7.27% 

86 

96.5 

113.1 

113.1 

76.04% 

-12.21% 

90.1 

99.9 

111.3 

111.3 

80.95% 

-10.88% 

98.9 

108.6 

116.1 

116.1 

85.19% 

-9.81% 

100.3 

111.5 

114.7 

114.7 

87.45% 

-11.17% 

102.5 

112.4 

115.4 

115.4 

88.82% 

-9.66% 

j- - 

105.6 

116.4 

115.8 

115.8 

91.19% 

-10.23% 

i 

j  K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

1  12 

1 

AF 

AIRCRAFT 

FSD 

ALLOC 

FPIF 

. -  -  - 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

68.10 

68.4 

163.5 

163.5 

41.65% 

-0.00 

,i 

87.10 

87.2 

163.5 

163.5 

53.27% 

-0.11% 

i 

97.8 

99.1 

163.5 

163.5 

59.82% 

-1.33% 

i 

103.1 

106.1 

163.5 

163.5 

63.06% 

-2.91% 

107.7 

111.8 

167.5 

167.5 

64.30% 

-3.81% 

j: 

112.9 

116.4 

167.5 

167.5 

67.40% 

-3.10% 

!: 

118.4 

123 

167.5 

167.5 

70.69% 

-3.89% 

\ 

i; 

123.2 

127.3 

168.7 

168.7 

73.03% 

-3.33% 

j; 

127.9 

131.7 

168.7 

168.7 

75.82% 

-2.97% 

\- 

1- 

130.9 

135.3 

168.7 

168.7 

77.59% 

-3.36% 

|; 

133.3 

136.8 

169.8 

169.8 

78.50% 

-2.63% 

136.8 

138.4 

169,6 

169.6 

80.66% 

-1.17% 

1; 

141.8 

143.9 

169.7 

169.7 

83.56% 

-1 .48% 

i; 

i: 

147.7 

149 

169.7 

169.7 

87.04% 

-0.88% 

E 

151.1 

151.9 

170.1 

170.1 

88.83% 

-0.53% 

153.9 

154.3 

170.1 

170.1 

90.48% 

-0.26% 

153 

155.3 

170.1 

170.1 

89.95% 

-1 .50% 

|  K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

(  13 

j 

f 

NAVY 

ELECTRONICS 

FSD 

ALLOC 

FP 

J— . 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

I 

j 

22.90 

23 

878.2 

878.2 

2.61% 

-0.00 

j 

50.40 

49.3 

923.3 

923.3 

5.46% 

2.18% 

j 

80.3 

78 

886 

886 

9.06% 

2.86% 

< 

112.4 

108.3 

886 

886 

12.69% 

3.65% 

f 

141.1 

145.2 

886.1 

886.1 

15.92% 

-2.91% 

181.2 

189.3 

886.1 

886.1 

20.45% 

-4.47% 

i 

222.5 

239.3 

886.1 

886.1 

25.11% 

-7.55% 

i 

264.8 

293.2 

886.1 

886.1 

29.88% 

-10.73% 

| 

305.7 

350.6 

886.1 

886.1 

34.50% 

-14.69% 

359.5 

412.6 

896.8 

896.8 

40.09% 

-14.77% 

413 

471.3 

891.1 

891.1 

46.35% 

-14.12% 

472 

535.3 

902.2 

902.2 

52.32% 

-13.41% 

71 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

14 

ARMY 

AIRCRAFT 

FSD 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

1.10 

1.2 

43 

43 

2.56% 

-0.09 

4.70 

5.1 

43.3 

43.3 

10.85% 

-8.51% 

7.9 

10.1 

43.3 

43.3 

18.24% 

-27.85% 

13.2 

16.5 

43.3 

43.3 

30.48% 

-25.00% 

19.9 

24.3 

43.3 

43.3 

45.96% 

-22.11% 

25.9 

32.9 

43.4 

43.4 

59.68% 

-27.03% 

31.1 

40.3 

43.4 

43.4 

71 .66% 

-29.58% 

32.6 

44.5 

44.2 

44.2 

73.76% 

-36.50% 

34 

47.3 

44.2 

44.2 

76.92% 

-39.12% 

35.9 

50.9 

45.1 

45.1 

79.60% 

-41 .78% 

40.8 

55.6 

45.1 

45.1 

90.47% 

-36.27% 

41.9 

58 

45.2 

45.2 

92.70% 

-38.42% 

42.7 

60.4 

45.2 

45.2 

94.47% 

-41 .45% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

15 

ARMY 

AIRCRAFT 

DEV 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

0.40 

0.3 

34.7 

34.7 

1.15% 

0.25 

1.20 

1.1 

34.7 

34.7 

3.46% 

8.33% 

13 

12.8 

34.7 

34.7 

37.46% 

1.54% 

17.5 

17.4 

34.7 

34.7 

50.43% 

0.57% 

22.2 

22.1 

34.7 

34.7 

63.98% 

0.45% 

26.6 

26.3 

40.2 

40.2 

66.17% 

1.13% 

29.8 

29.9 

40.2 

40.2 

74.13% 

-0.34% 

32.9 

33 

40.2 

40.2 

81 .84% 

-0.30% 

35.7 

34.8 

40.5 

40.5 

88.15% 

2.52% 

37.4 

36.4 

40.5 

40.5 

92.35% 

2.67% 

38.7 

37.8 

40.8 

40.8 

94.85% 

2.33% 

38.9 

38.4 

40.8 

40.8 

95.34% 

1 .29% 

39 

38.6 

40.8 

40.8 

95.59% 

1.03% 

39.2 

39 

40.8 

40.8 

96.08% 

0.51% 

39.6 

39.5 

40.8 

40.8 

97.06% 

0.25% 

40.2 

40.1 

40.8 

40.8 

98.53% 

0.25% 

40.6 

40.6 

41 

41 

99.02% 

0.00% 

K  #: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

16 

ARMY 

AIRCRAFT 

DEV 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

3.10 

3.2 

58.6 

58.6 

5.29% 

-0.03 

10.60 

11.4 

58.9 

58.9 

18.00% 

-7.55% 

16.8 

20.3 

58.9 

58.9 

28.52% 

-20.83% 

24.5 

29 

58.9 

58.9 

41.60% 

-18.37% 

32.6 

38.1 

58.9 

58.9 

55.35% 

-16.87% 

39.2 

46.7 

59.1 

59.1 

66.33% 

-19.13% 

44.7 

55.6 

59.2 

59.2 

75.51% 

-24.38% 

46.7 

60.1 

60.4 

60.4 

77.32% 

-28.69% 

48.2 

63.1 

60.4 

60.4 

79.80% 

-30.91  % 

50.6 

66.9 

61.3 

61.3 

82.54% 

-32.21% 

56.8 

74.2 

61.4 

61.4 

92.51  % 

-30.63% 

72 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

17 

AF 

AIRCRAFT 

FSD 

ALLOC 

FPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL PT 

CV 

53.40 

52.7 

2523.1 

2523.1 

2.12% 

0.01 

79.70 

79.5 

2523.1 

2523.1 

3.16% 

0.25% 

128.7 

125.5 

2523.1 

2523.1 

5.10% 

2.49% 

163.9 

160.6 

2523.1 

2523.1 

6.50% 

2.01% 

235.8 

229.8 

3006.7 

3006.7 

7.84% 

2.54% 

274.3 

266 

3006.7 

3006.7 

9.12% 

3.03% 

433.2 

427.5 

3656.7 

3656.7 

11.85% 

1.32% 

551.2 

551.8 

3678.3 

3678.3 

14.99% 

-0.11% 

719.2 

737.2 

3678.3 

3678.3 

19.55% 

-2.50% 

1272.3 

1361.8 

3778.9 

3778.9 

33.67% 

-7.03% 

1580.9 

1729.3 

3778.9 

3778.9 

41.83% 

-9.39% 

2077.8 

2415.1 

3778.9 

3778.9 

54.98% 

-16.23% 

2584.1 

3260.8 

3775.4 

3775.4 

68.45% 

-26.19% 

2933 

4371.1 

3775.4 

3775.4 

77.69% 

-49.03% 

3065.7 

4443.7 

3774.9 

3774.9 

81.21% 

-44.95% 

3382.7 

4923.4 

3779 

3779 

89.51% 

-45.55% 

351 1 .8 

5100.3 

3783 

3783 

92.83% 

-45.23% 

3597.3 

5234 

3782.8 

3782.8 

95.10% 

-45.50% 

3660.7 

5346.2 

3789.6 

3789.6 

96.60% 

-46.04% 

3695.8 

5417.4 

3789.5 

3789.5 

97.53% 

-46.58% 

3725.8 

5490 

3789.5 

3789.5 

98.32% 

-47.35% 

3771.3 

5650 

3820.6 

3820.6 

98.71% 

-49.82% 

5153.3 

8096.9 

5551.3 

5551.3 

92.83% 

-57.12% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

KTYPE: 

18 

NAVY 

AIRCRAFT 

DEV 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

32.70 

36.6 

71.9 

71.9 

45.48% 

-0.12 

34.30 

42.3 

77.8 

77.8 

44.09% 

-23.32% 

37.6 

44.4 

78.4 

78.4 

47.96% 

-18.09% 

53.2 

60.7 

71 .8 

71 .8 

74.09% 

-14.10% 

55.8 

64.4 

74 

74 

75.41% 

-15.41% 

58.1 

68.1 

76 

76 

76.45% 

-17.21% 

63.7 

71.6 

79.2 

79.2 

80.43% 

-12.40% 

68 

76.7 

80.2 

80.2 

84.79% 

-12.79% 

71.5 

80.4 

80.7 

80.7 

88.60% 

-12.45% 

73.7 

84 

82.7 

82.7 

89.12% 

-13.98% 

75.8 

86.5 

83.2 

83.2 

91.11% 

-14.12% 

76.4 

86.1 

84 

84 

90.95% 

-12.70% 

77.4 

87.6 

84.1 

84.1 

92.03% 

-13.18% 

77.5 

87.9 

84.2 

84.2 

92.04% 

-13.42% 

73 


13301 

1984 

1389.8 

1985.2 

SYSTEM: 

AIRCRAFT 

PHASE: 

DEV 

ACWP 

CBB 

543.8 

763.9 

556.5 

764.7 

619.8 

765.2 

660.3 

774.2 

697 

782.5 

732.1 

790.2 

753.3 

793.9 

777.1 

795.3 

800.4 

803.1 

824.6 

824.1 

844.4 

895.2 

856.4 

889.1 

884.4 

929.4 

899.2 

940 

907.3 

938.3 

922.7 

939.6 

69.43% 

70.80% 

78.23% 

81.83% 

84.63% 

87.32% 

88.70% 

91.09% 

92.33% 

92.74% 

87.58% 

89.58% 

87.56% 

87.90% 

88.84% 

90.41% 


-0.03 

-2.79% 

-3.54% 

-4.23% 

-5.26% 

-6.10% 

-6.97% 

-7.27% 

-7.94% 

-7.89% 

-7.70% 

-7.52% 

-8.68% 

-8.82% 

-8.84% 

-8.62% 


K#: 

21 

SERVICE: 

AF 

BCWP 

SYSTEM: 

AIRCRAFT 

ACWP 

PHASE: 

DEV 

CBB 

ALLOC 

BUDGET 

K  TYPE: 

CP 

COMPL  PT 

CV 

2.30 

2.7 

66.4 

66.4 

3.46% 

-0.17 

6.30 

6.6 

65.9 

65.9 

9.56% 

-4.76% 

8.4 

9.5 

83.3 

83.3 

10.08% 

-13.10% 

13.8 

16.3 

90.4 

90.4 

15.27% 

-18.12% 

28.9 

28.8 

121.8 

121.8 

23.73% 

0.35% 

48.3 

48.3 

196.4 

196.4 

24.59% 

0.00% 

56.3 

55.7 

216.8 

216.8 

25.97% 

1.07% 

71.2 

70.4 

297.5 

297.5 

23.93% 

1.12% 

81.7 

80.4 

297 

297 

27.51% 

1.59% 

93.1 

91.8 

197.6 

197.6 

47.12% 

1.40% 

101.9 

100.3 

197.5 

197.5 

51 .59% 

1.57% 

115.5 

114.4 

199.1 

199.1 

58.01% 

0.95% 

132.1 

131.5 

202.5 

202.5 

65.23% 

0.45% 

147.7 

147.8 

205.2 

205.2 

71.98% 

-0.07% 

159.8 

160.6 

203.2 

203.2 

78.64% 

-0.50% 

170.9 

173.1 

202.8 

202.8 

84.27% 

-1 .29% 

179.8 

184.5 

201.9 

201.9 

89.05% 

-2.61% 

185.5 

192.4 

215.7 

215.7 

86.00% 

-3.72% 

190.1 

200.4 

217.6 

217.6 

87.36% 

-5.42% 

193.9 

205.9 

218.4 

218.4 

88.78% 

-6.19% 

197.2 

203.7 

218.5 

218.5 

90.25% 

-3.30% 

203.3 

215.6 

220.9 

220.9 

92.03% 

-6.05% 

K#: 

22 

SERVICE: 

AF 

BCWP 

SYSTEM: 

AIRCRAFT 

ACWP 

PHASE: 

FSD 

CBB 

ALLOC 

BUDGET 

K  TYPE: 
CPAF 

COMPL  PT 

CV 

189.40 

187.1 

9241 .8 

9241.8 

2.05% 

0.01 

395.10 

391.8 

9244 

9244 

4.27% 

0.84% 

641.5 

647.5 

9248.9 

9248.9 

6.94% 

-0.94% 

725.6 

732 

9248.9 

9248.9 

7.85% 

-0.88% 

822.1 

840.6 

9248.7 

9248.7 

8.89% 

-2.25% 

1188.9 

1208.3 

10275.7 

10275.7 

11.57% 

-1.63% 

1503.2 

1539.1 

10286.8 

10286.8 

14.61% 

-2.39% 

1839.2 

1857.1 

10302.3 

10302.3 

17.85% 

-0.97% 

2192.5 

2215 

10313.4 

10313.4 

21 .26% 

-1.03% 

2525 

2548.8 

10415 

10415 

24.24% 

-0.94% 

2873.8 

2944.1 

10415.8 

10415.8 

27.59% 

-2.45% 

3238.7 

3308.9 

10525.3 

10525.3 

30.77% 

-2.17% 

3599 

3713.8 

10524.5 

10524.5 

34.20% 

-3.19% 

3949.1 

4084.5 

10578.6 

10578.6 

37.33% 

-3.43% 

4319.2 

4515.2 

10586.8 

10586.8 

40.80% 

-4.54% 

75 


SERVICE: 

AF 


SYSTEM: 

AIRCRAFT 


SERVICE: 

NAVY 


SYSTEM: 

AIRCRAFT 


PHASE: 

FSD 


1322.2 

1322.2 

1322.2 

1322.2 

1437.4 

1461.6 
1463.9 

1462.6 

1470.5 
1655.1 

1752.5 

1761.5 

1761.7 

1761.8 
PHASE: 

DEV 


ALLOC 

BUDGET 

K  TYPE: 
CPAF 

COMPL  PT 

CVi 

* 

1322.2 

1322.2 

8.41% 

13.52% 

o.o: 

-2.6£ 

l 

5% 

1322.2 

19.71% 

-1.85 

5% 

1322.2 

23.56% 

-2.57 

'% 

1437.4 

28.19% 

-5.11 

% 

1461.6 

32.43% 

-5.02 

% 

1463.9 

37.06% 

-3.12 

:% 

1462.6 

40.68% 

-3.18 

% 

1470.5 

43.54% 

-3.84 

% 

1655.1 

42.03% 

-4.08 

% 

1752.5 

43.12% 

-3.61 

% 

1761.5 

46.50% 

-3.24 

% 

1761.7 

50.32% 

-2.48 

% 

1761.8 

54.26% 

-3.21 

% 

ALLOC 

BUDGET 


K  TYPE: 
CP 

COMPL  PT 


12.41% 

16.87% 

24.07% 

30.66% 

35.83% 

40.87% 

46.92% 

53.18% 

60.96% 

67.60% 

73.89% 

83.09% 

88.93% 

89.12% 

92.61% 

94.80% 

96.79% 

95.26% 

96.29% 

96.38% 

96.73% 

97.04% 


76 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

25 

NAVY 

AIRCRAFT 

DEV 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

21.80 

20.6 

1020.2 

1020.2 

2.14% 

0.06 

51.90 

49.5 

1022.2 

1022.2 

5.08% 

4.62% 

95.3 

97 

1047.2 

1047.2 

9.10% 

-1 .78% 

117.2 

123.4 

1048.2 

1048.2 

11.18% 

-5.29% 

168.9 

185.9 

1055.1 

1055.1 

16.01% 

-10.07% 

236.8 

260.3 

1079.1 

1079.1 

21.94% 

-9.92% 

372.2 

412.2 

1092.2 

1092.2 

34.08% 

-10.75% 

462.2 

515.6 

1103 

1103 

41.90% 

-11.55% 

527.1 

597.3 

1105.4 

1105.4 

47.68% 

-13.32% 

546.3 

639.1 

1105.4 

1281.5 

42.63% 

-16.99% 

580.9 

674.8 

1109.7 

1281.7 

45.32% 

-16.16% 

644.3 

748 

1115.9 

1288 

50.02% 

-16.09% 

713.4 

850.5 

1148.2 

1320.2 

54.04% 

-19.22% 

771 

933.4 

1152.9 

1324.9 

58.19% 

-21.06% 

836.6 

1011.3 

1167.5 

1339.5 

62.46% 

-20.88% 

876.4 

1071.2 

1186.5 

1358.6 

64.51% 

-22.23% 

928.6 

1150.7 

1185.9 

1357.9 

68.39% 

-23.92% 

970.4 

1207.9 

1185.9 

1375.6 

70.54% 

-24.47% 

941.7 

1271.1 

1184 

1590 

59.23% 

-34.98% 

961.5 

1294.1 

1184 

1590 

60.47% 

-34.59% 

1049.8 

1340.9 

1185.5 

1502.1 

69.89% 

-27.73% 

1044.2 

1396.1 

1186.8 

1590 

65.67% 

-33.70% 

1075.1 

1458.2 

1186.7 

1590 

67.62% 

-35.63% 

1077.6 

1502 

1189.7 

1632 

66.03% 

-39.38% 

1085 

1516.7 

1189.7 

1632 

66.48% 

-39.79% 

1098.8 

1547.5 

1189.7 

1632 

67.33% 

-40.84% 

1113.8 

1582.3 

1189.7 

1632.2 

68.24% 

-42.06% 

1130.7 

1616.8 

1189.9 

1631.1 

69.32% 

-42.99% 

1144 

1640.2 

1189.9 

1631.1 

70.14% 

-43.37% 

1165.1 

1670.1 

1193.7 

1636 

71.22% 

-43.34% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

26 

NAVY 

AIRCRAFT 

FSD 

ALLOC 

CPAF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

30.70 

28.7 

3436.5 

3436.5 

0.89% 

0.07 

66.80 

61.5 

3436.5 

3436.5 

1.94% 

7.93% 

171.2 

163.7 

3290.5 

3290.5 

5.20% 

4.38% 

267 

250.9 

3290.5 

3290.5 

8.11% 

6.03% 

370.1 

355.7 

3292.1 

3292.1 

11.24% 

3.89% 

509.6 

509.4 

3274.1 

3274.1 

15.56% 

0.04% 

670.7 

678.8 

3321 .7 

3321 .7 

20.19% 

-1.21% 

885.5 

877.5 

3333.6 

3333.6 

26.56% 

0.90% 

970 

956.9 

3373.6 

3373.6 

28.75% 

1.35% 

1052.5 

1038.5 

3371 .2 

3371 .2 

31.22% 

1.33% 

1134.3 

1116.7 

3371 .2 

3371 .2 

33.65% 

1 .55% 

1371.7 

1363.2 

3372 

3372 

40.68% 

0.62% 

1597.4 

1598.5 

3401.8 

3401.8 

46.96% 

-0.07% 

1857.4 

1871.7 

3402 

3402 

54.60% 

-0.77% 

77 


%  N. 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

30 

NAVY 

TORPEDO 

FSD 

ALLOC 

CPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

3.4 

3.5 

434.7 

434.7 

0.78% 

-2.94% 

49.9 

51 .6 

427 

427 

11.69% 

-3.41% 

108 

113.3 

431.1 

431.1 

25.05% 

-4.91% 

150.3 

161.7 

433.6 

433.6 

34.66% 

-7.58% 

204.8 

225.4 

433.9 

433.9 

47.20% 

-10.06% 

255.6 

291.6 

436.2 

436.2 

58.60% 

-14.08% 

336 

341.1 

517.8 

517.8 

64.89% 

-1.52% 

352.3 

358.7 

520.3 

520.3 

67.71% 

-1.82% 

460.9 

458.9 

663.3 

663.3 

69.49% 

0.43% 

527.3 

528.8 

666.7 

666.7 

79.09% 

-0.28% 

584 

592.9 

668.6 

668.6 

87.35% 

-1.52% 

627.8 

647.5 

670.3 

670.3 

93.66% 

-3.14% 

649.3 

681.1 

671.5 

671.5 

96.69% 

-4.90% 

661.6 

703.1 

669 

669 

98.89% 

-6.27% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

31 

ARMY 

MISSILE 

FSD 

ALLOC 

CPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

10.7 

9.5 

17.8 

17.8 

60.11% 

11.21% 

17.9 

15.3 

43.4 

43.4 

41 .24% 

14.53% 

24.6 

22.8 

50.7 

50.7 

48.52% 

7.32% 

36.2 

34 

89.3 

89.3 

40.54% 

6.08% 

50.6 

46.5 

93.5 

93.5 

54.12% 

8.10% 

63.3 

58.1 

94.4 

94.4 

67.06% 

8.21% 

74.3 

68.4 

95.3 

95.3 

77.96% 

7.94% 

83.9 

78.9 

95.6 

95.6 

87.76% 

5.96% 

89.8 

86.2 

99.4 

99.4 

90.34% 

4.01% 

91.8 

89 

99.4 

99.4 

92.35% 

3.05% 

92.8 

90.1 

99.4 

99.4 

93.36% 

2.91% 

94.1 

91.3 

100.4 

100.4 

93.73% 

2.98% 

95.2 

91.9 

100.4 

100.4 

94.82% 

3.47% 

96.2 

93.6 

101.3 

101.3 

94.97% 

2.70% 

96.8 

95.1 

101.3 

101.3 

95.56% 

1.76% 

98 

96.3 

101.3 

101.3 

96.74% 

1.73% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

32 

ARMY 

MISSILE 

FSD 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

76.6 

74.1 

507.4 

507.4 

15.10% 

3.26% 

204.8 

204.4 

507.1 

507.1 

40.39% 

0.20% 

409.6 

411 

548.2 

548.2 

74.72% 

-0.34% 

463.6 

470.6 

552 

552 

83.99% 

-1.51% 

477 

485.9 

551.8 

551.8 

86.44% 

-1.87% 

507.3 

517.8 

570.4 

570.4 

88.94% 

-2.07% 

541.7 

554.2 

954.2 

954.2 

56.77% 

-2.31% 

603.3 

614.8 

955.3 

955.3 

63.15% 

-1.91% 

671.2 

696.9 

956.8 

956.8 

70.15% 

-3.83% 

739.7 

756.7 

1041.7 

1041.7 

71.01% 

-2.30% 

810.4 

829.6 

1039.3 

1039.3 

77.98% 

-2.37% 

884.9 

906 

1062.8 

1062.8 

83.26% 

-2.38% 

961.2 

991.1 

1097.3 

1097.3 

87.60% 

-3.11% 

1023.9 

1066.1 

1106.9 

1106.9 

92.50% 

-4.12% 

1071.9 

1127.6 

1106.3 

1106.3 

96.89% 

-5.20% 

79 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

33 

ARMY 

MISSILE 

FSD 

ALLOC 

CPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

73 

82.2 

156.8 

156.8 

46.56% 

-12.60% 

94.4 

112.9 

155.1 

155.1 

60.86% 

-19.60% 

146.4 

147.9 

155.1 

155.1 

94.39% 

-1.02% 

165.6 

169.4 

253.1 

253.1 

65.43% 

-2.29% 

194.8 

194.8 

309.1 

309.1 

63.02% 

0.00% 

223 

223.2 

309.5 

309.5 

72.05% 

-0.09% 

248.4 

260.7 

309.5 

309.5 

80.26% 

-4.95% 

273 

302 

309.5 

309.5 

88.21% 

-10.62% 

279 

326.4 

309.5 

309.5 

90.15% 

-16.99% 

279 

338.6 

347.3 

347.3 

80.33% 

-21.36% 

279 

359.5 

371.4 

371.4 

75.12% 

-28.85% 

401 

401.6 

426.8 

426.8 

93.96% 

-0.15% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

34 

NAVY 

MISSILE 

FSD 

ALLOC 

CPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

7.9 

6.6 

188 

188 

4.20% 

16.46% 

18.2 

15.9 

188 

188 

9.68% 

12.64% 

28.3 

28.9 

187.8 

187.8 

15.07% 

-2.12% 

40.4 

43.9 

187.8 

187.8 

21.51% 

-8.66% 

53.6 

58.9 

187.9 

1879 

28.53% 

-9.89% 

67.6 

76.4 

1887 

188.7 

35.82% 

-13.02% 

81.2 

94.1 

189.9 

189.9 

42.76% 

-15.89% 

95.7 

112.4 

191.3 

191.3 

50.03% 

-17.45% 

107.6 

127.8 

191.6 

191.6 

56.16% 

-18.77% 

122.8 

146.2 

194.5 

194.5 

63.14% 

-19.06% 

135.3 

157  8 

195 

195 

69.38% 

-16.63% 

146.4 

173.6 

282.4 

282.4 

51 .84% 

-18.58% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

35 

NAVY 

MISSILE 

FSD 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

28.9 

31.6 

55 

55 

52.55% 

-9.34% 

35 

38 

66.9 

66.9 

52.32% 

-8.57% 

41.7 

46.4 

66.6 

66.6 

62.61% 

-1 1 .27% 

52.3 

54.2 

94.5 

94.5 

55.34% 

-3.63% 

55.5 

62 

76.4 

76.4 

72.64% 

-11.71% 

58.1 

71.2 

77.6 

84.6 

68.68% 

-22.55% 

59.5 

78.4 

77.1 

86.1 

69.11% 

-31 .76% 

61.7 

83.9 

76.5 

86.5 

71 .33% 

-35.98% 

65.2 

89.3 

78.4 

88.4 

73.76% 

-36.96% 

60.3 

89.7 

79.4 

95.4 

63.21% 

-48.76% 

60.5 

89.9 

81.9 

97.9 

61 .80% 

-48.60% 

60.3 

91.3 

76.5 

92.5 

65.19% 

-51 .41  % 

74.3 

93.1 

77.8 

93.8 

79.21% 

-25.30% 

80 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

36 

NAVY 

MISSILE 

FSD 

ALLOC 

FPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

445.8 

489.8 

1534 

1534 

29.06% 

-9.87% 

548  4 

612 

1534 

1534 

35.75% 

-11.60% 

685.1 

784.9 

1534 

1534 

44.66% 

-14.57% 

809.5 

950.8 

1537.6 

1537.6 

52.65% 

-17.46% 

919.9 

1071.8 

1537.6 

1537.6 

59.83% 

-16.51% 

1052.5 

1266.5 

1545.7 

1545.7 

68.09% 

-20.33% 

1161.1 

1431 

1546.4 

1546.4 

75.08% 

-23.25% 

1254.3 

1552.3 

1547.6 

1547.6 

81.05% 

-23.76% 

1296.6 

1643.5 

1547.6 

1941 

66.80% 

-26.75% 

1345.3 

1729.9 

1554.4 

1941 

69.31% 

-28.59% 

1377.8 

1813.2 

1547 

1941 

70.98% 

-31.60% 

1397.8 

1861.6 

1547 

1940.5 

72.03% 

-33.18% 

1405 

1885.6 

1547 

1940.7 

72.40% 

-34.21% 

1419.2 

1936.1 

1547 

1941 

73.12% 

-36.42% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

KTYPE: 

37 

NAVY 

MISSILE 

FSD 

ALLOC 

CPAF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

64.3 

65.6 

550 

550 

11.69% 

-2.02% 

115.8 

120 

550 

550 

21 .05% 

-3.63% 

180.3 

190.7 

557.7 

557.7 

32.33% 

-5.77% 

258.2 

268.3 

557.8 

557.8 

46.29% 

-3.91% 

393.4 

311.7 

557.8 

557.8 

70.53% 

20.77% 

512.8 

522.6 

2400.5 

2400.5 

21.36% 

-1.91% 

652.6 

674.1 

2397 

2397 

27.23% 

-3.29% 

832.1 

863.6 

2394.4 

2394.4 

34.75% 

-3.79% 

960.1 

996.7 

2394.4 

2394.4 

40.10% 

-3.81% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

KTYPE: 

38 

NAVY 

MISSILE 

FSD 

ALLOC 

CPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

15 

11.2 

65 

65 

23.08% 

25.33% 

23.3 

21.6 

128.9 

128.9 

18.08% 

7.30% 

38.6 

26.6 

131.6 

131.6 

29.33% 

31.09% 

36 

33.9 

133.4 

133.4 

26.99% 

5.83% 

43 

41.4 

133.4 

133.4 

32.23% 

3.72% 

57.2 

57.2 

132.6 

132.6 

43.14% 

0.00% 

67.1 

68.4 

130.8 

130.8 

51.30% 

-1 .94% 

72.1 

74.8 

133.4 

133.4 

54.05% 

-3.74% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

39 

ARMY 

VEHICLE 

FSD 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

1.8 

1.6 

44.7 

44.7 

4.03% 

11.11% 

9.5 

8.8 

44.7 

44.7 

21.25% 

7.37% 

17.3 

17.8 

44.7 

44.7 

38.70% 

-2.89% 

24.6 

26.2 

44.7 

44.7 

55.03% 

-6.50% 

31 

33.8 

49.1 

49.1 

63.14% 

-9.03% 

35.3 

40.2 

52.3 

52.3 

67.50% 

-13.88% 

41.1 

45.8 

53.4 

59.1 

69.54% 

-1 1 .44% 

43.4 

52.5 

53.4 

59.8 

72.58% 

-20.97% 

44.2 

57.3 

53.4 

61 .2 

72.22% 

-29.64% 

47.2 

61.3 

53.2 

60.9 

77.50% 

-29.87% 

49.4 

64.8 

54.6 

62.4 

79.17% 

-31.17% 

52.1 

67.8 

56.5 

65.3 

79.79% 

-30.13% 

53.7 

68.5 

57 

64.7 

83.00% 

-27.56% 

81 


CM 


SERVICE 

AF 


45.6 

109.1 

150.6 

230.2 

297.8 

342.8 

412.9 

456.5 

500.1 

531.9 

554.2 

579.9 

586.5 
SERVICE: 

NAVY 


SYSTEM: 

AIRCRAFT 


46.3 

114.2 

163.1 

246.2 
314.6 

363 

433.9 

481.8 
526.1 
558.4 

585.9 

619.9 
630 

SYSTEM: 

ELECTRONICS 


PHASE 

DEV 


434.6 

430.3 

439.5 

553.6 

561.1 

567.9 
581 

589.7 

609.9 

644.9 
650.6 
666.5 

666.1 
PHASE: 

FSD 


ALLOC 

BUDGET 


96.80% 

96.43% 


K  TYPE: 
FPI 


COMPL  PT 

10.49% 

25.35% 

34.27% 

41 .58% 
53.07% 
60.36% 

71 .07% 
77.41% 
82.00% 
82.48% 
85.18% 
87.01% 
88.05% 

K  TYPE: 

CPIF 


>1 .54% 
-4.67% 
-8.30% 
-6.95% 
-5.64% 
-5.89% 
-5.09% 
-5.54% 
-5.20% 
-4.98% 
-5.72% 
-6.90% 
-7.42% 


ALLOC 


BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

5.4 

22.4 

292.1 

292.1 

1.85% 

-314.81% 

39.8 

42.3 

292.1 

292.1 

13.63% 

-6.28% 

50.1 

58.4 

308.3 

308.3 

16.25% 

-16.57% 

79.2 

80.7 

308.3 

308.3 

25.69% 

-1 .89% 

99.5 

100.5 

308.6 

308.6 

32.24% 

-1.01% 

113.7 

125.3 

369.8 

369.8 

30.75% 

-10.20% 

135.2 

147.1 

369.8 

369.8 

36.56% 

-8.80% 

171.7 

182.3 

388.2 

388.2 

44.23% 

-6.17% 

186.1 

208.8 

388.2 

388.2 

47.94% 

-12.20% 

193.9 

220.8 

379.3 

379.3 

51.12% 

-13.87% 

216.8 

248.8 

387.1 

387.1 

56.01% 

-14.76% 

229.8 

276.7 

387.1 

387.1 

59.36% 

-20.41% 

243.8 

299 

385.2 

385.2 

63.29% 

-22.64% 

225.1 

291 

386.9 

386.9 

58.18% 

-29.28% 

240.9 

313.9 

387.1 

387.1 

62.23% 

-30.30% 

259.2 

337.7 

386.3 

386.3 

67.10% 

-30.29% 

281 

366.9 

389.8 

389.8 

72.09% 

-30.57% 

296.4 

390 

389.5 

389.5 

76.10% 

-31 .58% 

321.2 

420.5 

390.7 

390.7 

82.21  % 

-30.92% 

342 

439.6 

390,3 

390.3 

87.62% 

-28.54% 

366.6 

469.6 

390.7 

390.7 

93.83% 

-28  10% 

82 


K#: 

43 

SERVICE: 

AF 

BCWP 

SYSTEM: 

MISSILE 

ACWP 

PHASE: 

DEV 

CBB 

ALLOC 

BUDGET 

K  TYPE: 
FPIF 

COMPL  PT 

CV 

3.6 

2.7 

87.9 

87.9 

4.10% 

25.00% 

21.5 

23.1 

119.1 

119.1 

18.05% 

-7.44% 

39.3 

43 

119.1 

119.1 

33.00% 

-9.41% 

52.2 

57.2 

119.2 

119.2 

43.79% 

-9.58% 

65.1 

74.4 

120.1 

120.1 

54.20% 

-14.29% 

78.1 

90.6 

122.4 

122.4 

63.81% 

-16.01% 

89.5 

106.1 

130.2 

130.2 

68.74% 

-18.55% 

98.1 

120 

132.9 

132.9 

73.81% 

-22.32% 

105.2 

135.4 

133.6 

133.6 

78.74% 

-28.71% 

109.8 

144.6 

133.3 

133.3 

82.37% 

-31 .69% 

117.1 

155 

133.6 

133.6 

87.65% 

-32.37% 

119.5 

166.4 

133.7 

133.7 

89.38% 

-39.25% 

K  #: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE* 

44 

AF 

SATELLITE 

FSD 

FPIF 

ALLOC 

- - - 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

100 

132.8 

570.7 

570.7 

17.52% 

-32.80% 

108.5 

148.2 

570.7 

570.7 

19.01% 

-36.59% 

125.9 

177.7 

572.9 

572.9 

21.98% 

-41.14% 

150.4 

219.2 

581.6 

581 .6 

25.86% 

-45.74% 

288.3 

450.2 

593.8 

593.8 

48.55% 

-56.16% 

337 

523.2 

611.4 

611.4 

55.12% 

-55.25% 

388.3 

549.7 

656.4 

656.4 

59.16% 

-41.57% 

458.6 

629.2 

731.7 

731.7 

62.68% 

-37.20% 

536.5 

722.1 

737.6 

737.6 

72.74% 

-34.59% 

609.7 

816.5 

747.9 

747.9 

81.52% 

-33.92% 

629.1 

866.6 

757.6 

757.6 

83.04% 

-37.75% 

651 .2 

909.5 

795.2 

795.2 

81.89% 

-39.67% 

689.9 

969.6 

802.2 

802.2 

86.00% 

-40.54% 

729.2 

995.3 

824 

1114.3 

65.44% 

-36.49% 

760.1 

1046.9 

821.7 

1115.5 

68.14% 

-37.73% 

804.3 

1062.3 

870.3 

1119.8 

71.83% 

-32.08% 

820.4 

1088.5 

873.9 

1158.5 

70.82% 

-32.68% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

45 

NAVY 

ELECTRONICS 

FSD 

CP 

ALLOC 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

1.1 

0.6 

80.7 

80.7 

1.36% 

45.45% 

4.2 

3.7 

80.7 

80.7 

5.20% 

11.90% 

10.2 

9.5 

80.7 

80.7 

12.64% 

6.86% 

17.7 

17.2 

81.4 

81.4 

21.74% 

2.82% 

26.3 

25.8 

85.5 

85.5 

30.76% 

1.90% 

36.3 

39.2 

100.7 

100.7 

36.05% 

-7.99% 

45 

54 

100.7 

100.7 

44.69% 

-20.00% 

47.5 

58.4 

101 

101 

47.03% 

-22.95% 

82.3 

81.7 

147 

147 

55.99% 

0.73% 

92.6 

93 

147 

147 

62.99% 

-0.43% 

104.4 

109.3 

147 

147 

71.02% 

-4.69% 

107.2 

116.5 

147 

147 

72.93% 

-8.68% 

118.1 

132.4 

147 

147 

80.34% 

-12.11% 

123.1 

146.9 

147 

147 

83.74% 

-19.33% 

126.7 

152.2 

147 

147 

86.19% 

-20.13% 

83 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

46 

AF 

MISSILE 

FSD 

ALLOC 

FPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

27.1 

27.8 

441 

441 

6.15% 

-2.58% 

43.6 

43.9 

483.2 

483.2 

9.02% 

-0.69% 

67.3 

66 

126.2 

126.2 

53.33% 

1 .93% 

81.3 

80.8 

119.7 

119.7 

67.92% 

0.62% 

93.5 

96.7 

119.7 

119.7 

78.11% 

-3.42% 

106.3 

106.8 

119.7 

119.7 

88.81% 

-0.47% 

115.6 

115.5 

137.7 

137.7 

83.95% 

0.09% 

118.9 

119.6 

137.7 

137.7 

86.35% 

-0.59% 

123.9 

124.9 

139.9 

139.9 

88.56% 

-0.81% 

129.5 

129.9 

138.4 

138.4 

93.57% 

-0.31% 

135 

135.1 

166.5 

166.5 

81.08% 

-0.07% 

138.9 

139.5 

166.4 

166.4 

83.47% 

-0.43% 

146.3 

147.2 

166.4 

166.4 

87.92% 

-0.62% 

160.3 

161.1 

172.1 

172.1 

93.14% 

-0.50% 

168.9 

169.3 

172.1 

172.1 

98.14% 

-0.24% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

47 

ARMY 

MISSILE 

DEV 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

0.01 

3.7 

86.4 

86.4 

0.01% 

NA 

8.7 

10.1 

84.4 

84.4 

10.31% 

-16.09% 

16.8 

20.9 

93.5 

93.5 

17.97% 

-24.40% 

79.6 

80.7 

163.2 

163.2 

48.77% 

-1 .38% 

93.6 

90.5 

163.6 

163.6 

57.21% 

3.31% 

109.9 

111.1 

163.7 

163.7 

67.14% 

-1 .09% 

130.7 

131.9 

164.5 

164.5 

79.45% 

-0.92% 

143 

146 

170.8 

170.8 

83.72% 

-2.10% 

151.1 

156.6 

169.7 

169.7 

89.04% 

-3.64% 

156.8 

163.5 

170.5 

170.5 

91.96% 

-4.27% 

160.2 

167.8 

172.4 

172.4 

92.92% 

-4.74% 

163.9 

171.4 

172.8 

172.8 

94.85% 

-4.58% 

166.2 

175.6 

174.2 

174.2 

95.41% 

-5.66% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

48 

ARMY 

MISSILE 

DEV 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

0.5 

0.5 

43.5 

43.5 

1.15% 

0.00% 

2 

2 

43.5 

43.5 

4.60% 

0.00% 

4.5 

4.5 

43.5 

43.5 

10.34% 

0.00% 

12.2 

11.4 

43.5 

43.5 

28.05% 

6.56% 

21.8 

21.9 

43.6 

43.6 

50.00% 

-0.46% 

27.8 

28.6 

49.9 

49.9 

55.71% 

-2.88% 

35.7 

37.6 

50 

50 

71 .40% 

-5.32% 

41.4 

45.7 

50.5 

50.5 

81.98% 

-10.39% 

47.1 

53.4 

63 

63 

74.76% 

-13.38% 

52.2 

61.8 

62.8 

62.8 

83.12% 

-18.39% 

55.2 

67.9 

62.9 

62.9 

87.76% 

-23  01% 

60 

73 

66.9 

66.9 

89.69% 

-21 .67% 

62.3 

77.9 

66 

66 

94.39% 

-25.04% 

64.7 

78.3 

66.8 

78.1 

82.84% 

-21 .02%  1 

84 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

49 

AF 

AIRCRAFT 

DEV 

ALLOC 

FPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COM  PL  PT 

CV 

15 

15.9 

100.3 

100.3 

14.96% 

-6.00% 

22.1 

24 

101.1 

101.1 

21 .86% 

-8.60% 

29 

34.8 

101.1 

101.1 

28.68% 

-20.00% 

35 

45.8 

101.1 

101.1 

34.62% 

-30.86% 

50.6 

80.5 

101 .8 

101.8 

49.71% 

-59.09% 

75.2 

103.9 

105.7 

143 

52.59% 

-38.16% 

81.6 

114.7 

109 

146.3 

55.78% 

-40.56% 

87 

130.8 

109 

146.3 

59.47% 

-50.34% 

91 .6 

148.6 

109 

146.3 

62.61% 

-62.23% 

94.2 

170.6 

107.3 

146.7 

64.21% 

-81.10% 

100.6 

200.4 

106.4 

146.7 

68.58% 

-99.20% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

KTYPE: 

50 

AF 

AIRCRAFT 

DEV 

ALLOC 

FPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

14 

14 

108.9 

108.9 

12.86% 

0.00% 

21.6 

21 

108.9 

108.9 

19.83% 

2.78% 

33.4 

33.4 

108.9 

108.9 

30.67% 

0.00% 

38.1 

39.4 

108.9 

108.9 

34.99% 

-3.41% 

47.1 

48.6 

108.9 

108.9 

43.25% 

-3.18% 

56.3 

58.5 

108.9 

108.9 

51.70% 

-3.91% 

68.3 

72.4 

112.7 

109.7 

62.26% 

-6.00% 

74.9 

84.8 

112.7 

112.7 

66.46% 

-13.22% 

82.8 

95.3 

111.4 

111.4 

74.33% 

-15.10% 

87.9 

103.4 

111.7 

111.4 

78.90% 

-17.63% 

95.7 

112.9 

112.3 

112.3 

85.22% 

-17.97% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

KTYPE: 

51 

NAVY 

ELECTRONICS 

DEV 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

1.1 

1.1 

80.8 

80.8 

1.36% 

0.00% 

6.2 

5.3 

80.8 

80.8 

7.67% 

14.52% 

10.4 

11.1 

80.8 

80.8 

12.87% 

-6.73% 

21.3 

26.7 

88.1 

88.1 

24.18% 

-25.35% 

28.6 

40.2 

88.1 

88.1 

32.46% 

-40.56% 

73.6 

84.2 

88.7 

88.7 

82.98% 

-14.40% 

63.7 

103.6 

88.7 

88.7 

71.82% 

-62.64% 

68.1 

116.9 

88.7 

88.7 

76.78% 

-71 .66% 

74.9 

139.4 

88.8 

88.8 

84.35% 

-86.11% 

77 

148.8 

88.8 

88.8 

86.71% 

-93.25% 

113.5 

168.1 

128.3 

128.3 

88.46% 

-48.11% 

112.8 

165 

128.3 

128.3 

87.92% 

-46.28% 

85 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

52 

ARMY 

MISSILE 

DEV 

ALLOC 

FPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL PT 

CV 

20.4 

20.3 

162.7 

162.7 

12.54% 

0.49% 

27.8 

27.6 

162.7 

162.7 

17.09% 

0.72% 

31.5 

32.3 

162.8 

162.8 

19.35% 

-2.54% 

36.4 

37.1 

162.8 

162.8 

22.36% 

-1 .92% 

41.4 

41.8 

162.8 

162.8 

25.43% 

-0.97% 

45.3 

45.6 

162.8 

162.8 

27.83% 

-0.66% 

50.4 

49.7 

163 

163 

30.92% 

1.39% 

55.1 

55.2 

163 

163 

33.80% 

-0.18% 

59.1 

59.8 

163 

163 

36.26% 

-1.18% 

70.6 

71.7 

163 

163 

43.31% 

-1 .56% 

92.4 

96.1 

163.3 

163.3 

56.58% 

-4.00% 

99.6 

104.5 

164.5 

164.5 

60.55% 

-4.92% 

112.4 

117.3 

165.1 

165.1 

68.08% 

-4.36% 

122.2 

130.3 

165.1 

165.1 

74.02% 

-6.63% 

132.4 

141.3 

165.1 

165.1 

80.19% 

-6.72% 

142 

152.4 

165.1 

165.1 

86.01% 

-7.32% 

149.8 

162.7 

165.1 

165.1 

90.73% 

-8.61% 

155.3 

171.1 

165.1 

165.1 

94.06% 

-10.17% 

159.4 

178.6 

165.1 

165.1 

96.55% 

-12.05% 

163.4 

178.5 

165.2 

165.2 

98.91% 

-9.24% 

164.2 

180.5 

165.2 

165.2 

99.39% 

-9.93% 

164.7 

181.5 

165.2 

165.2 

99.70% 

-10.20% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

53 

ARMY 

MISSILE 

DEV 

ALLOC 

CPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

39.1 

41.3 

343.5 

343.5 

1 1 .38% 

-5.63% 

64.3 

68.8 

343.5 

343.5 

18.72% 

-7.00% 

92.2 

102.6 

343.5 

343.5 

26.84% 

-1 1 .28% 

121 

137.4 

343.5 

343.5 

35.23% 

-13.55% 

132.9 

150.7 

343.5 

343.5 

38.69% 

-13.39% 

184.2 

184.2 

343.5 

343.5 

53.62% 

0.00% 

226.8 

231.5 

343.5 

343.5 

66.03% 

-2.07% 

251.1 

257.9 

343.5 

343.5 

73.10% 

-2.71% 

269.8 

280.1 

422.8 

422.8 

63.81% 

-3.82% 

'  281 .4 

291.7 

422.8 

422.8 

66.56% 

-3.66% 

305.2 

318.4 

422.8 

422.8 

72.19% 

-4.33% 

315.1 

330.8 

422.8 

422.8 

74.53% 

-4.98% 

359 

359 

518.3 

518.3 

69.26% 

0.00% 

377.8 

377.7 

499 

499 

75.71% 

0.03% 

_ _ _ _ _ 

390.9 

388.9 

499 

499 

78.34% 

0.51% 
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%  in 


K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

54 

AF 

ELECT 

DEV 

ALLOC 

CPAF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

0.8 

0.8 

71 

71 

1.13% 

0.00% 

1.6 

1.6 

71 

71 

2.25% 

0.00% 

2.7 

2.5 

72.1 

72.1 

3.74% 

7.41% 

4.4 

4.2 

72.1 

72.1 

6.10% 

4.55% 

5.6 

5.7 

72.1 

72.1 

7.77% 

-1.79% 

7.9 

8.2 

72.1 

72.1 

10.96% 

-3.80% 

10.7 

11.5 

72.1 

72.1 

14.84% 

-7.48% 

13.1 

14.8 

72.1 

72.1 

18.17% 

-12.98% 

15.3 

17.6 

72.1 

72.1 

21.22% 

-15.03% 

17.7 

21.8 

72.1 

72.1 

24.55% 

-23.16% 

20.5 

26.3 

72.1 

72.1 

28.43% 

-28.29% 

23 

29.7 

72.1 

72.1 

31.90% 

-29.13% 

26.4 

34.6 

72.1 

87.9 

30.03% 

-31.06% 

29 

39 

72.1 

87.9 

32.99% 

-34.48% 

33.5 

46.1 

72.1 

87.9 

38.11% 

-37.61% 

35.2 

49.7 

72.6 

88.3 

39.86% 

-41.19% 

37.7 

54.1 

72.6 

88.3 

42.70% 

-43.50% 

39.1 

57.2 

72.6 

88.3 

44.28% 

-46.29% 

40.9 

61.1 

72.6 

88.3 

46.32% 

-49.39% 

50.1 

81.3 

72.8 

88.7 

56.48% 

-62.28% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

55 

ARMY 

MISSILE 

DEV 

ALLOC 

CP 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

11.3 

11.6 

43.6 

43.6 

25.92% 

-2.65% 

15.4 

15.6 

43.9 

43.9 

35.08% 

-1.30% 

20.1 

21.1 

49.3 

49.3 

40.77% 

-4.98% 

26 

27.1 

50.2 

50.2 

51 .79% 

-4.23% 

31.1 

34.2 

51.5 

51.5 

60.39% 

-9.97% 

35.4 

40.6 

54.6 

54.6 

64.84% 

-14.69% 

37.4 

43 

54.9 

54.9 

68.12% 

-14.97% 

47.8 

55.9 

59.2 

65.4 

73.09% 

-16.95% 

54.8 

60.9 

60.9 

65.4 

83.79% 

-11.13% 

60.7 

67.5 

70.8 

77.5 

78.32% 

-1 1 .20% 

64.4 

74.9 

67.5 

72 

89.44% 

-16.30% 

66.3 

77 

67.8 

71.8 

92.34% 

-16.14% 

66.4 

78 

67.5 

72 

92.22% 

-17.47% 
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K  #: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

56 

AF 

SUPPORT 

DEV 

ALLOC 

FPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL PT 

CV 

102.3 

96.4 

128.1 

128.1 

79.86% 

5.77% 

103.2 

97 

128.1 

128.1 

80.56% 

6.01% 

104.3 

98.1 

128.4 

128.4 

81 .23% 

5.94% 

105.1 

98.6 

128.5 

128.5 

81.79% 

6.18% 

105.9 

99.4 

128.6 

128.6 

82.35% 

6.14% 

107.4 

100.7 

129.5 

129.5 

82.93% 

6.24% 

109.3 

101.7 

129.9 

129.9 

84.14% 

6.95% 

110.1 

102.2 

129.9 

129.9 

84.76% 

7.18% 

112.6 

104.2 

131.7 

131.7 

85.50% 

7.46% 

115.9 

107.6 

131.7 

131.7 

88.00% 

7.16% 

117.6 

109.3 

131.8 

131.8 

89.23% 

7.06% 

119.9 

111.4 

134 

134 

89.48% 

7.09% 

121.7 

112.5 

135.8 

135.8 

89.62% 

7.56% 

123.3 

113.9 

134.2 

134.2 

91.88% 

7.62% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

57 

AF 

MISSILE 

DEV 

ALLOC 

FPI 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

16.1 

16.3 

30.7 

30.7 

52.44% 

-1.24% 

18.9 

18.9 

32.5 

32.5 

58.15% 

0.00% 

21.6 

21.8 

33.5 

33.5 

64.48% 

-0.93% 

25.2 

25.1 

34.3 

34.3 

73.47% 

0.40% 

27.2 

27.1 

34.3 

34.3 

79.30% 

0.37% 

29 

28.7 

34.3 

34.3 

84.55% 

1 .03% 

32.1 

32.4 

34.9 

34.9 

91.98% 

-0.93% 

33.1 

33.7 

38.1 

38.1 

86.88% 

-1.81% 

K#: 

SERVICE: 

SYSTEM: 

PHASE: 

K  TYPE: 

58 

ARMY 

MISSILE 

DEV 

ALLOC 

CPIF 

BCWP 

ACWP 

CBB 

BUDGET 

COMPL  PT 

CV 

30.4 

30.8 

238.8 

238.8 

12.73% 

-1.32% 

33.1 

32.9 

238.8 

238.8 

13.86% 

0.60% 

38.1 

38.7 

238.8 

238.8 

15.95% 

-1.57% 

44.2 

45.8 

238.8 

238.8 

18.51% 

-3.62% 

50.2 

51.9 

238.8 

238.8 

21.02% 

-3.39% 

56.8 

59.6 

238.8 

238.8 

23.79% 

-4.93% 

62.5 

65.7 

238.8 

238.8 

26.17% 

-5.12% 

69.6 

74.2 

238.8 

238.8 

29.15% 

-6.61% 

88.9 

96.7 

238.8 

238.8 

37.23% 

-8.77% 

94.9 

109 

236.8 

272.5 

34.83% 

-14.86% 

112.1 

129.8 

236.8 

272.5 

41.14% 

-15.79% 

128.3 

151.1 

236.8 

272.5 

47.08% 

-17.77% 

145.1 

173.2 

236.8 

272.5 

53.25% 

-19.37% 

159.6 

193.6 

237.9 

273.6 

58.33% 

-21.30% 

175.7 

214.8 

240.5 

276.2 

63.61% 

-22.25% 

206.2 

260.1 

241.1 

289.5 

71.23% 

-26.14% 

218.4 

277.8 

241.1 

289.5 

75.44% 

-27.20% 

226.3 

291 

241.4 

289.8 

78.09% 

-28.59% 

230.7 

301.2 

241.4 

289.8 

79.61  % 

-30.56% 

234.9 

309.5 

241 .7 

290.3 

80.92% 

-31 .76% 
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APPENDIX  B.  STATISTICAL  INFORMATION 


This  appendix  provides  the  detailed  statistical  information  to  support  information 
contained  in  the  text.  The  appendix  is  subdivided  into  the  following  sections: 

A.  Modeling  of  the  Data. 

B.  Distribution  Information 

C.  Beta  Value  Conversions 

D.  Non-parametric  Tests 
A.  MODELING  OF  THE  DATA 

The  accuracy  of  the  A- 12  percentile  values  is  dependent  on  how  well  the  data  were 
modeled  by  the  specified  parameters.  To  ensure  that  the  percentile  values  were  not  biased 
by  the  choice  of  distribution,  each  data  set  was  checked  with  a  second  distribution.  This 
section  contains  the  graphical  representations  of  the  distribution  of  the  data.  The  graphs 
in  this  section  were  produced  using  BestFit  [Ref.  20], 

1.  Sample:  0  to  10  Percent  Complete 

Figure  7  shows  the  distribution  of  cost  variances  for  all  contracts  from  0  to  10 
percent  complete.  The  data  were  best  modeled  by  the  Normal  distribution,  with  mean 
equal  to  1.79  and  standard  deviation  of  1 1.56. 

Comparison  of  Input  Distribution  and  Normal(1. 79, 11.56) 


Input 
Normal 

"“-2.9  -2.3  -1.7  -1.1  -0.6  0.0  0^6  1/1  i.7  2.3  2.9 

Values  in  10A1 

Figure  7.  Sample  Data  0  to  10  Percent  Complete. 
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2.  Sample  Data:  11  to  20  Percent  Complete. 

Figure  8  shows  the  distribution  for  cost  variances  of  the  sample,  from  1 1-20 
percent  complete.  Thirty-seven  contracts  were  included  in  the  sample.  The  data  were 
skewed  to  the  left,  and  were  best  represented  by  the  Beta  distribution  with  parameters 


Figure  8.  Sample  Data  1 1  to  20  Percent  Complete. 

3.  Sample  Data:  21  to  30  Percent  Complete 

Figure  9  is  the  distribution  of  cost  variances  for  contracts  21  to  30  percent 


complete.  Thirty-four  contracts  constitute  the  sample.  The  data  were  centered  near  zero 


and  were  modeled  by  the  Logistic  distribution  with  parameters  (-5.27,  11.10). 


4.  Sample  Data  31  to  40  Percent  Complete 


Figure  10  is  the  distribution  of  cost  variances  in  programs  31  to  40  percent 
complete.  Thirty-seven  contracts  constitute  the  sample.  The  sample  is  modeled  by  the 
Logistic  distribution  with  parameters  (-8.08,  5.54). 


Figure  10.  Sample  Data  31  to  40  Percent  Complete. 

5.  Sample  Data  41  to  50  Percent  Complete 

Figure  1 1  is  the  distribution  of  cost  variances  in  programs  41  to  50  percent 
complete.  The  data  are  skewed  to  the  left.  However,  the  degree  of  skewness  does  not 
warrant  the  selection  of  a  Beta  distribution.  The  Logistic  distribution  models  the  data. 
The  parameters  of  the  distribution  are  (-1 1.02,  8.08). 


6.  Aircraft  Programs:  0  to  10  Percent  Complete 


Figure  12  provides  the  distribution  of  cost  variances  within  aircraft  programs,  from 
0  to  1 0  percent  complete.  Eleven  contracts  are  used.  The  distribution  is  centered  near 
zero.  The  data  are  modeled  by  the  Beta  distribution  with  parameters  (1 .75,  .83). 


Figure  12.  Aircraft  Programs  0  to  10  Percent  Complete. 

7.  Aircraft  Programs:  11  to  20  Percent  Complete 

Figure  13  provides  the  distribution  of  cost  variances  in  aircraft  programs  1 1  to  20 


percent  complete.  The  Beta  distribution  modeled  the  data.  Eleven  contracts  were  in  the 


sample.  The  parameters  of  the  distribution  are  ( 1 . 1 7,  .56). 


Figure  13.  Aircraft  Programs  11  to  20  Percent  Complete. 
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8.  Aircraft  Programs:  21  to  30  Percent  Complete 

Figure  14  provides  the  cost  variance  distribution  for  aircraft  programs  21  to  30 
percent  complete.  The  Beta  distribution  models  the  data  with  parameters  (1 .43,  .64). 
Twelve  contracts  were  included  in  the  sample. 


Comparison  of  Input  Distribution  and  Beta(1.43,0.64)  *  15.13 

+ -14.30 


Figure  14.  Aircraft  Programs  21  to  30  Percent  Complete. 

9.  Aircraft  Programs:  31  to  40  Percent  Complete 
Figure  15  provides  the  distribution  of  cost  variances  in  aircraft  programs  3 1  to  40 
percent  complete.  The  data  are  skewed  to  the  left  and  are  modeled  by  the  Beta 
distribution.  Fourteen  contracts  were  included  in  the  sample.  The  parameters  of  the 
distribution  are  (2.05,  .94). 

Comparison  of  Input  Distribution  and  Beta(2.05,0.94)  * 

32.30  +  -30.86 


Input 


Beta 


-3.09  -2.47  -1.85  -1.23  -0.62  0.00  0.62  1.23  1.85 

Values  in  10A1 


Figure  15.  Aircraft  Programs  31  to  40  Percent  Complete. 
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10.  Aircraft  Programs:  41  to  50  Percent  Complete 


Figure  16  provides  the  distribution  of  cost  variances  within  aircraft  programs  41 
to  50  percent  complete.  The  data  are  modeled  by  the  Logistic  distribution  with 
parameters  (-12.63,  10. 16).  Ten  contracts  were  included  in  the  sample. 


Figure  16.  Aircraft  Programs  41  to  50  Percent  Complete. 

11.  Sample  Contracts  Cost  Variance  Distribution 

Figure  17  provides  the  distribution  of  cost  variances  for  all  programs  that  had  met 


or  exceeded  80  percent  complete.  Forty  contracts  were  included  in  the  sample.  The 


distribution  was  skewed  to  the  left  and  was  modeled  by  the  Beta  distribution.  The 


parameters  of  the  distribution  are  (1.49,  .82). 


Figure  17.  All  Contracts  at  Least  80  Percent  Complete. 


12.  Distribution  of  Cost  Contracts 

Figure  18  provides  the  distribution  of  cost  variances,  for  programs  with  cost-type 
contracts,  at  least  50  percent  complete.  The  data  are  skewed  to  the  left  and  are  modeled 
by  the  Beta  distribution.  Thirty-nine  contracts  were  included  in  the  sample.  The 
parameters  of  the  distribution  are  (1.65,  .40). 


Figure  18.  All  Cost-Type  Contracts  at  Least  50  Percent  Complete. 

13.  Distribution  of  Fixed-Price  Contracts 


Figure  19  provides  the  cost  variances  for  programs  with  fixed-price  type  contracts 


that  are  at  least  50  percent  complete.  The  number  of  contracts  included  in  the  sample  is 
15.  The  parameters  of  the  Beta  distribution  are  (  .95,  .22). 


Figure  19.  All  Fixed-Price  Contracts  at  Least  50  Percent  Complete. 
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15.  Distribution  of  Contract  Budget  Base  Adjustments 

Figure  20  shows  the  distribution  of  adjustments  to  the  Contract  Budget  Base,  for 
all  programs.  The  programs  included  had  to  be  at  least  70  percent  complete  and  have  50 
percent  of  the  total  program  included  in  the  cost  reports.  The  distribution  of  CBB 
adjustments  is  modeled  by  the  Logistic  distribution  with  parameters  (53.57,  30.21). 
Twenty-nine  contracts  were  included  in  the  sample. 


B.  DISTRIBUTION  INFORMATION 


1.  Statistics  of  the  Distributions 

Table  21  gives  the  detailed  statistics  of  the  distributions  that  best  modeled  the  data. 
Column  1  is  the  type  sample  that  is  modeled.  Column  2  provides  the  percentage 
completion  brackets.  In  Columns  3  and  4  are  the  distribution's  parameters.  In  Columns  5 
and  6  are  the  extreme  values  for  the  data.  The  extreme  values  do  not  include  outliers  that 
were  discarded.  The  best  distribution  that  modeled  the  data  and  the  second  best 
distribution  are  given  in  Columns  7  and  8.  The  accompanying  Chi-squared  values  are 
provided  in  Columns  9  and  10. 
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SAMPLE 

% 

COMPL 

PARAM 

1 

PARAM 

2 

Hi 

LO 

BEST 

DISTRO 

2ND  BEST 
DISTRO 

BEST 

XA2 

2ND  BEST 
XA2 

:  0-10 

11,56 

2&68 

-25 

Normal 

Logistic 

17,56 

17.13 

ALL 

11-20 

2.03 

1.07 

6.86 

-34.69 

Beta 

Logistic 

16.55 

19.89 

ALL 

21 ,30: 

-527 

6,06 

20.75 

^Logistic:* 

is  Normal 

26,64 

40.36 

ALL 

31-40 

-6.08 

5.54 

3.72 

-40.56 

Logistic 

Normal 

26.04 

33.94 

:  ALL  •••". 

41-50 

-11,02 

6,06 

9,31 

-5909 

Logistic 

mm 

74.01 

AC  FT 

0-10 

1.75 

0.83 

6.34 

-22.22 

Beta 

Normal 

9.04 

10.17 

11-20 

117 

0.56 

1.97 

-27.65 

Normal 

llipill 

8.86 

ACFT 

21  -30 

1.43 

0.64 

0.83 

-1 4.3 

Beta 

Normal 

9.69 

14.18 

mm  mm 

IMMM 

2:05  J 

0.94  . 

1.44 

-30.86 

1762 

24.59 

ACFT 

41  -50 

-12.63 

10.16 

1.4 

-59.09 

Logistic 

Beta 

20.51 

5.64 

>50%;:; 

-22.30 

15.76 

7.62 

-99.2 

Logistic 

18.76 

5.68 

COST  ONLY 

>50% 

.........  1 ,65 

0.4 

1.73 

-62.28 

....  Beta ...... 

Normal 

5.76 

33.16 

•:::xX:::ALL::,, 

1.49 

Mmm 

IliiiS 

.  -57.12 

^  X&g&te  :  ■ 

21.34  i 

ALL  CBB 

NA 

53.67 

30.21 

232.68 

-60.97 

Logistic 

Normal 

12.19 

18.43 

Table  21.  Detailed  Distribution  Information. 


2.  Percentile  Value  Information 

Table  22  contains  the  percentile  values  for  the  samples  based  on  the  best  and 
second  best  distributions.  Columns  1  and  2  give  the  sample  type  and  the  percentage 
completion  bracket.  Columns  3  and  4  provide  the  0  to  1  Beta  value,  if  applicable.  In 
Columns  5  and  6  are  the  .01  percentile  and  .05  percentile  values  for  the  given  distribution. 
The  next  best  percentile  values  are  given  in  Columns  7  and  8. 


SAMPLE 

% 

COMPL 

.05 

0  to  1 
BETA 

.01 

0  tol 
BETA 

BEST 

0.05 

BEST 

0.01 

2ND 

BEST 

0.05 

2ND 

BEST 

0.01 

ALL 

0-10 

-17.22 

-17.13 

>-19:88 

ALL 

11-20 

0,2600 

0.1300 

-23.85 

-29.33 

21  -30 

-23.17 

>33.21 

-22.03 

! -28:47,, 

ALL 

31-40 

-24.39 

-33.53 

-23.52 

-31.09 

ALL  :: 

41-50 

-34-81 

:  >48:1 5: 

-24,72 

-31.62 

ACFT 

0-10 

0.2100 

0  0800 

-16.33 

-19.85 

-37.52 

-47.73 

ACFT 

11  -  20 

0.1300 

0.0300 

-24:00 

-26:86: 

-17.50 

1-23.29 

ACFT 

21-30 

0.1800 

0.0600 

-11.65 

-13.42 

-18.40 

-23.48 

ACFT  : ; 

31-40 

0.2400 

0.1100 

-23  06 

-27:29 

-15.71 

ACFT 

41  -  50 

-42.54 

-59.32 

-43.14 

-55,78 

FP  ONLY 

>50%; 

-47:79 

-5T75 

-68.79 

-94.81 

COST  ON 

>50% 

0.3030 

01194 

-42.73 

-54.52 

-40.80 

-51.77 

all  m 

>80% 

0.1566 

6  0536 

-46:98 

-53:65 

-37  09 

ALL  CBB 

NA 

142.51 

192.39 

144.30 

181.69 

Table  22.  Percentile  Value  Information. 
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C.  BETA  VALUE  CONVERSIONS 

The  Beta  distribution  is  a  continuous  distribution,  with  range  0  to  1 .  Cost 
variances  can  vary  from  negative  infinity  to  positive  infinity.  To  model  cost  variances 

using  a  Beta  distribution  the  observed  values  must  be  converted  to  a  range  of  0  to  1  This 

is  accomplished  by  using  the  following  formula:  b( P:  v,  >r)  +  tf[l-(P:  v,  vr)]  Where 

a  >  x  >  b  and  (£:  v,w)  =  J u^x (1  -  ii)^~xdn.  [Ref.  12] 

o 

The  observed  cost  variances  tended  to  assume  one  of  two  shapes.  If  the 
distribution  was  skewed  to  the  left,  with  a  majority  of  the  observations  clustering  close  to 
zero,  then  the  distribution  assumed  a  J  shape,  characteristic  of  the  Beta  distribution  with 
(v-l)(w-l)  <  0.  If  a  relatively  large  number  of  observations  occurred  in  the  tail  of  the 
distribution  then  the  distribution  tended  to  assume  a  U  shape  where  v  <  1  and  w  <  1 . 

The  estimated  parameters  provided  by  BestFit  were  used  to  obtain  the  percentile 
values.  The  percentile  values  were  then  converted  to  values  ranging  from  a  <  x  <  b  by 
arithmetic  manipulation  of  the  original  formula. 

The  distributions  shown  in  Section  A,  have  a  Beta  distribution  ranging  from  0  to  1 
superimposed  over  data  ranging  from  a  to  b. 

D.  NON-PARAMETRIC  TESTS  OF  THE  HYPOTHESES 

1.  The  Mann-Whitney  Test 

The  Mann-Whitney  Test  tests  for  differences  between  the  means  of  two 
populations.  The  test  statistic  is  given  by:  U  -  \xul<3u  =  Z.  For  a  two  tailed  test  the 
decision  criterion  is:  Reject  H0  if  Z  >  -za,2  or  Z  <  zii/2.  For  all  four  tests  alpha  is  set  at  .05. 

Therefore,  a/2  equals  .025.  The  value  of  z^  equals  1.96. 

2.  Hypothesis  2 

Pjp  =  |j.c.  This  hypothesis  tests  for  differences  in  the  means  of  fixed-price  contracts 
and  the  mean  of  cost-type  contracts.  Hg,  the  null  hypothesis,  is  the  mean  cost  variance  of 

fixed-price  contracts  equals  the  mean  of  cost-type  contracts.  HA,  the  alternative 
hypothesis,  is  the  means  are  not  equal.  This  is  a  two  tailed  test.  The  purpose  of  this  test 
is  to  determine  if  the  choice  of  contract  type  contributed  to  the  exceptional  cost  overruns 
in  the  A- 12  program. 

A  requirement  of  the  Mann-Whitney  test  is  the  populations  must  be  the  same, 
except  for  possible  locations  of  the  mean.  The  standard  deviations  of  cost-type  contracts 
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and  fixed-price  type  contracts  was  significantly  different:  28.23  and  16.13  respectively. 

To  standardize  the  two  populations,  each  set  of  data  was  divided  by  its  standard  deviation 
before  conducting  the  test.  The  test  statistics  for  Hypothesis  2  are  as  follows: 


From  the  results  of  the  test  the  null  hypothesis  can  not  be  rejected.  There  is  no 
statistical  difference  in  the  means  of  the  two  populations.  The  test  statistics  for  this  test 
are  given  below: 


R1 

R2 

nl 

n2 

Test 

Statistic 

381 

1104 

15 

39 

0.6083 

3.  Hypothesis  3 

^acft  =  M' sample*  Hypothesis  3  tests  for  differences  in  the  cost  variances  of  aircraft 
contracts  and  other  programs.  This  test  examines  if  aircraft  programs  experience 

statistically  different  cost  variances  than  other  type  of  programs.  The  mean  cost  variance 
of  aircraft  programs  is  -14.03  with  standard  deviation  15.69.  The  mean  cost  variance  of 
other  programs  in  the  sample  is  -14. 10  with  standard  deviation  16.43.  No  difference  was 
assumed  between  the  two  populations.  As  would  be  expected,  with  similar  means  and 
standard  deviations,  the  null  hypothesis  could  not  be  rejected.  The  test  statistics  are  given 
below: 


R1 

R2 

nl 

n2 

Test 

Statistic 

324 

711 

15 

30 

0.5056 

4.  Hypothesis  4 


M'cbbfp  M-cbbc'  Hypothesis  4  is  similar  to  Hypothesis  2.  Hypothesis  2  tested  for 
differences  in  the  cost  variances  between  fixed-price  type  and  cost-type  contracts. 

Hypothesis  4  tests  for  differences  in  the  mean  budget  adjustments.  The  mean  CBB 
adjustment  for  fixed-price  type  contracts  is  43.93  with  standard  deviation  59.84.  The 
mean  CBB  adjustment  for  cost-type  contracts  is  57.24  with  standard  deviation  54.37. 
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The  test  statistics,  for  Hypothesis  4,  are  given  below: 


Test 

R1 

R2 

nl 

n2 

Statistic 

115 

320 

8 

21 

0.244 

The  null  hypothesis  could  not  be  rejected.  There  is  no  statistical  difference 
between  the  mean  CBB  adjustments  based  on  contract  type. 

5.  Hypothesis  5. 

ftcbbacft  ftcbbsample'  Hypothesis  5  tests  for  differences  in  the  CBB  adjustments  in 
aircraft  programs  versus  other  types  of  programs.  The  mean  CBB  adjustment  for  aircraft 

programs  is  94. 15  with  standard  deviation  86. 15.  The  mean  CBB  adjustment  for  other 
types  of  programs  in  the  sample  is  42.99  with  standard  deviation  40.20.  The  test  statistics 
are  given  below: 


Test 

R1 

R2 

nl 

n2 

Statistic 

80 

355 

6 

23 

0.5384 

The  two  populations  were  divided  by  their  respective  standard  deviations  before 
the  test  to  eliminate  the  difference  in  variance  between  the  two  populations.  The  test 
results  indicate  that  the  null  hypothesis  can  not  be  rejected.  Despite  the  apparent 
differences  in  the  means  of  the  two  populations,  there  is  insufficient  evidence  to  claim  that 
aircraft  programs  mean  budget  adjustments  are  greater  than  other  programs. 
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